[LU-15064] sanity-sec test 58 fails with 'fail to ls' Created: 05/Oct/21 Updated: 17/Oct/21 Resolved: 17/Oct/21 |
|
| Status: | Resolved |
| Project: | Lustre |
| Component/s: | None |
| Affects Version/s: | Lustre 2.15.0 |
| Fix Version/s: | Lustre 2.15.0 |
| Type: | Bug | Priority: | Minor |
| Reporter: | James Nunez (Inactive) | Assignee: | Sebastien Buisson |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Severity: | 3 |
| Rank (Obsolete): | 9223372036854775807 |
| Description |
|
sanity-sec test_58 started failing with 'fail to ls' on 02 OCT 2021 for ARM client testing, https://testing.whamcloud.com/test_sets/d93471f3-2773-4a87-ae2c-016d85c16ed5 with the following in the suite_log == sanity-sec test 58: access to enc file's xattrs ======= 21:25:37 (1633209937)
10.9.5.158@tcp:/lustre /mnt/lustre lustre rw,flock,user_xattr,lazystatfs,encrypt 0 0
CMD: trevis-79vm3.trevis.whamcloud.com grep -c /mnt/lustre' ' /proc/mounts
Stopping client trevis-79vm3.trevis.whamcloud.com /mnt/lustre (opts:)
CMD: trevis-79vm3.trevis.whamcloud.com lsof -t /mnt/lustre
CMD: trevis-79vm3.trevis.whamcloud.com umount /mnt/lustre 2>&1
Starting client: trevis-79vm3.trevis.whamcloud.com: -o user_xattr,flock,test_dummy_encryption trevis-33vm1@tcp:/lustre /mnt/lustre
CMD: trevis-79vm3.trevis.whamcloud.com mkdir -p /mnt/lustre
CMD: trevis-79vm3.trevis.whamcloud.com mount -t lustre -o user_xattr,flock,test_dummy_encryption trevis-33vm1@tcp:/lustre /mnt/lustre
/mnt/lustre/d58.sanity-sec/f58.sanity-sec: count 1
0: pfid [0x20000dedd:0x1:0x0], name ':\xe4\x88\x8f\xf3g\xc8\x8eO\x9f\xa1\xa5\x8a\xb1\xa3G'
/mnt/lustre/d58.sanity-sec/subdir: count 1
0: pfid [0x20000dedd:0x1:0x0], name '\xd5-)=o\xebz\x13G\xb7\xc9\x8aD8%\x1d\xb6'
ls: cannot access '/mnt/lustre/racer/1/0': No such file or directory
sanity-sec test_58: @@@@@@ FAIL: fail to ls
and for regular x86_64 testing https://testing.whamcloud.com/test_sets/b35bcae6-b630-419a-82c9-b00e7971ce43 with == sanity-sec test 58: access to enc file's xattrs ======= 11:45:25 (1633175125)
10.2.8.139@tcp:/lustre /mnt/lustre lustre rw,flock,user_xattr,lazystatfs,encrypt 0 0
CMD: onyx-41vm5 grep -c /mnt/lustre' ' /proc/mounts
Stopping client onyx-41vm5 /mnt/lustre (opts:)
CMD: onyx-41vm5 lsof -t /mnt/lustre
CMD: onyx-41vm5 umount /mnt/lustre 2>&1
Starting client: onyx-41vm5: -o user_xattr,flock,test_dummy_encryption onyx-41vm8@tcp:/lustre /mnt/lustre
CMD: onyx-41vm5 mkdir -p /mnt/lustre
CMD: onyx-41vm5 mount -t lustre -o user_xattr,flock,test_dummy_encryption onyx-41vm8@tcp:/lustre /mnt/lustre
/mnt/lustre/d58.sanity-sec/f58.sanity-sec: count 1
0: pfid [0x20000cf38:0x1:0x0], name '\xd2`41\x8d#Z\x07\x81\x16\xc7\xe0\xf5\x9f\x15\x06'
/mnt/lustre/d58.sanity-sec/subdir: count 1
0: pfid [0x20000cf38:0x1:0x0], name '\xe87\xb8i\xd0N\x19\x05:)\xe8\x88>\x14\x1b\xa8'
ls: cannot access '/mnt/lustre/racer/12/9/1/18/10/12/19/7/18/6/10/5/13/10/17/0/2/19/10/15/19/12/0/19/15/13/2/7/18/16/12/15/3/18/10/15/13/10/11/9/5/15/15/10/18/2/4/8/19/15/0/8/7/8/4/0/4/18/19/4/10/16/16/12/7/2/18/15/11/8/4/17/0/11/12/19/13/9/17/19/10/5/13/17/1/0/9/13/2/10/14/10/12/1/10/10/5/17/1/10/0/17/1/3/6/13/11/8/2/9/4/16/12/7/14/8/7/0/10/1/8/4/2/9/5/8/7/8/8/15/9/7/9/13/1/7/2/9/11/10/7/19/16/14/14/4/13/19/8/15/9/3/1/11/3/6/7/15/11/16/5/6/1/6/16/11/10/5/19/19/19': Invalid argument
sanity-sec test_58: @@@@@@ FAIL: fail to ls
Trace dump:
= /usr/lib64/lustre/tests/test-framework.sh:6320:error()
= /usr/lib64/lustre/tests/sanity-sec.sh:4358:test_58()
ON client1 (vm5), I see the following in dmesg [21883.960890] Lustre: DEBUG MARKER: == sanity-sec test 58: access to enc file's xattrs ======= 11:45:25 (1633175125) [21884.196363] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts [21884.208378] Lustre: DEBUG MARKER: lsof -t /mnt/lustre [21884.343262] Lustre: DEBUG MARKER: umount /mnt/lustre 2>&1 [21884.384818] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre [21884.395117] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption onyx-41vm8@tcp:/lustre /mnt/lustre [21884.657373] bash (23201): drop_caches: 3 [21898.165028] bash (23201): drop_caches: 3 [21919.811108] LustreError: 26458:0:(lcommon_cl.c:196:cl_file_inode_init()) lustre: failed to initialize cl_object [0x20000bf82:0x11c3:0x0]: rc = -22 [21919.811118] LustreError: 26458:0:(llite_lib.c:2985:ll_prep_inode()) new_inode -fatal: rc -22 [21920.219193] Lustre: DEBUG MARKER: /usr/sbin/lctl mark sanity-sec test_58: @@@@@@ FAIL: fail to ls |
| Comments |
| Comment by Peter Jones [ 06/Oct/21 ] |
|
eb |
| Comment by Sebastien Buisson [ 06/Oct/21 ] |
|
jamesanunez I can see that this problem occurs only with the full-part-2 and full-dne-part-2 test sessions that you triggered. All other test instances pass sanity-sec test_58, so I am wondering what the differences are between these test sessions and the ones that are regularly triggered as part of patch testing. |
| Comment by Sebastien Buisson [ 07/Oct/21 ] |
|
sanity-sec test_58 most likely fails because of remnants from previous tests. I will update the test to avoid being exposed to that. |
| Comment by Gerrit Updater [ 07/Oct/21 ] |
|
"Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/45150 |
| Comment by Gerrit Updater [ 17/Oct/21 ] |
|
"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/45150/ |