[LU-297] llverfs: Open '/mnt/lustre/dir00007/file015' failed:Input/output error

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.1.0, Lustre 1.8.7
• Affects Version/s: Lustre 2.0.0, Lustre 1.8.6

Environment

Lustre Branch: b1_8
Lustre Build: http://newbuild.whamcloud.com/job/lustre-reviews/363/arch=x86_64,build_type=server,distro=el5,ib_stack=inkernel/
Test System: DDN SFA10000E (1Client+1MGS/MDT+1OST)
Network: tcp

• Severity: 3
• Bugzilla ID: 24,017
• Assignee: Jian Yu
• Reporter: Jian Yu

Description

While running llverfs in full mode on Lustre filesystem with 1TB OST, it hit -EIO error after writing (7*32+15)*4G=956G data into the filesystem as follows:

llverfs: Open '/mnt/lustre/dir00007/file015' failed:Input/output error

Dmesg log showed that:

Lustre: DEBUG MARKER: ======== run llverfs in full mode on the Lustre filesystem /mnt/lustre ========
LustreError: 21145:0:(mds_open.c:441:mds_create_objects()) error creating objects for inode 41943290: rc = -5
LustreError: 21145:0:(mds_open.c:826:mds_finish_open()) mds_create_objects: rc = -5

Before running llverfs, the Lustre disk space and inode usage info are as follows:

# lfs df -h /mnt/lustre
UUID                       bytes        Used   Available Use% Mounted on
largefs-MDT0000_UUID      224.0G      461.0M      210.7G   0% /mnt/lustre[MDT:0]
largefs-OST0000_UUID     1007.9G      472.6M      956.3G   0% /mnt/lustre[OST:0]

filesystem summary:      1007.9G      472.6M      956.3G   0% /mnt/lustre


# lfs df -i /mnt/lustre
UUID                      Inodes       IUsed       IFree IUse% Mounted on
largefs-MDT0000_UUID    58597923          26    58597897   0% /mnt/lustre[MDT:0]
largefs-OST0000_UUID    67108864          72    67108792   0% /mnt/lustre[OST:0]

filesystem summary:     58597923          26    58597897   0% /mnt/lustre

After running llverfs, the Lustre disk space and inode usage info are as follows:

# lfs df -h /mnt/lustre
UUID                       bytes        Used   Available Use% Mounted on
largefs-MDT0000_UUID      224.0G      461.0M      210.7G   0% /mnt/lustre[MDT:0]
largefs-OST0000_UUID     1007.9G      956.5G      276.8M 100% /mnt/lustre[OST:0]

filesystem summary:      1007.9G      956.5G      276.8M 100% /mnt/lustre


# lfs df -i /mnt/lustre
UUID                      Inodes       IUsed       IFree IUse% Mounted on
largefs-MDT0000_UUID    58598163         274    58597889   0% /mnt/lustre[MDT:0]
largefs-OST0000_UUID    13492969         328    13492641   0% /mnt/lustre[OST:0]

filesystem summary:     58598163         274    58597889   0% /mnt/lustre

Maloo report: https://maloo.whamcloud.com/test_sets/4b6fc288-7a43-11e0-a1b3-52540025f9af

Issue Links

• Trackbacks: Lustre 1.8.x known issues tracker While testing against Lustre b18 branch, we would hit known bugs which were already reported in Lustre Bugzilla https://bugzilla.lustre.org/. In order to move away from relying on Bugzilla, we would create a JIRA