[LU-6826] sanity test_71: No space left on device

• Type: Bug
• Resolution: Fixed
• Priority: Critical
• Fix Version/s: Lustre 2.8.0
• Affects Version/s: Lustre 2.8.0
• Labels: dne2
• Environment: server and client: lustre-master build # 3094 RHEL7
• Severity: 3
• Assignee: Di Wang
• Reporter: Maloo

Description

This issue was created by maloo for sarah_lw <[e-mail]>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/d8d84160-25eb-11e5-866a-5254006e85c2.

The sub-test test_71 failed with the following error:

dbench failed!

test log

== sanity test 71: Running dbench on lustre (don't segment fault) ==================================== 21:00:19 (1436389219)
copying /usr/share/dbench/client.txt to /mnt/lustre/d71.sanity/client.txt
copying necessary libs to /mnt/lustre/d71.sanity
lib64/libpopt.so.0
tar: lib64: Cannot mkdir: No space left on device
tar: lib64/libpopt.so.0: Cannot open: No such file or directory
lib64/libc.so.6
tar: lib64: Cannot mkdir: No space left on device
tar: lib64/libc.so.6: Cannot open: No such file or directory
lib64/ld-linux-x86-64.so.2
tar: lib64: Cannot mkdir: No space left on device
tar: lib64/ld-linux-x86-64.so.2: Cannot open: No such file or directory
tar: Exiting with failure status due to previous errors
status        script            Total(sec) E(xcluded) S(low) 
------------------------------------------------------------------------------------

test-framework exiting on error
 sanity test_71: @@@@@@ FAIL: dbench failed! 

Issue Links

• is related to: LU-6831 The ticket for tracking all DNE2 bugs

Comments

[Joseph Gmitter (Inactive)]

Landed for 2.8.0

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/16130/
Subject: LU-6826 lod: validate stripe_count and offset
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 46ebfdd558dbe57db6cf51351246ca81bd38e4c9

[Gerrit Updater]

wangdi ([e-mail]) uploaded a new patch: http://review.whamcloud.com/16130
Subject: LU-6826 lod: validate stripe_count and offset
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8e79c7c47dfc6e5a6230c440a88a8b26fa1cfe37

[Andreas Dilger]

The "-c1" case should be handled internally by the DNE kernel code to just create a non-striped directory. Doing anything else doesn't make sense.

[Oleg Drokin]

Ok, this problem actually does not have anything to do with out of space, but it's just DNE code allowing setdirstripe for more MDTs than we have in the system.
And then it fails mkdirs there:

[root@centos6-9 tests]# mkdir /mnt/lustre/test/dir
[root@centos6-9 tests]# rm -rf /mnt/lustre/test
[root@centos6-9 tests]# mkdir /mnt/lustre/test
[root@centos6-9 tests]# ../utils/lfs setdirstripe -D -c1 /mnt/lustre/test
[root@centos6-9 tests]# mkdir /mnt/lustre/test/dir
mkdir: cannot create directory `/mnt/lustre/test/dir': No space left on device

note that -CX where X is more than 1 are also accepted.

And coincidentally that's how test_71 looks like:

test_71() {
        test_mkdir -p $DIR/$tdir
        $LFS setdirstripe -D -c$MDSCOUNT $DIR/$tdir
        sh rundbench -C -D $DIR/$tdir 2 || error "dbench failed!"
}
run_test 71 "Running dbench on lustre (don't segment fault) ===="

So for non-DNE testcase this totally breaks the directory with such striping added

[Sarah Liu]

another instance hit in RHEL6.6 server/client:

https://testing.hpdd.intel.com/test_sets/a4e56756-2696-11e5-8b33-5254006e85c2