[LU-11681] sanity test 65i fails with 'find /mnt/lustre failed'

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.13.0
• Affects Version/s: Lustre 2.12.0, Lustre 2.13.0, Lustre 2.12.1, Lustre 2.12.2, Lustre 2.12.3, Lustre 2.12.4
• Labels: LTS12 dne zfs
• Environment: DNE/ZFS
• Severity: 3
• Assignee: Lai Siyao
• Reporter: James Nunez (Inactive)

Description

sanity test 65i is failing for DNE with ZFS configurations. Looking at the client test log at https://testing.whamcloud.com/test_sets/4ac91f66-e875-11e8-bfe1-52540065bddc , we see ‘lfs find’ is failing with

lfs find: llapi_semantic_traverse: Failed to open '/mnt/lustre/d60g.sanity/subdir2': No such file or directory (2)
error: find failed for /mnt/lustre.
 sanity test_65i: @@@@@@ FAIL: find /mnt/lustre failed 

Looking at the console logs, the only indication of a problem is in the client (vm7) console

 [ 3432.108946] Lustre: DEBUG MARKER: == sanity test 65i: various tests to set root directory striping ===================================== 23:16:02 (1542237362)
[ 3436.508827] LustreError: 662:0:(llite_lib.c:2390:ll_prep_inode()) new_inode -fatal: rc -2
[ 3438.077315] Lustre: DEBUG MARKER: /usr/sbin/lctl mark  sanity test_65i: @@@@@@ FAIL: find \/mnt\/lustre failed 

We’ve seen this test fail a total of three time since November 14, 2018.
https://testing.whamcloud.com/test_sets/da7b847a-ebb1-11e8-86c0-52540065bddc
https://testing.whamcloud.com/test_sets/5f393496-eba8-11e8-bfe1-52540065bddc

Issue Links

• is duplicated by: LU-11767 sanity test_60g: FAIL: mkdir failed
• is related to: LU-11907 sanity fails to cleanup d60g.sanity
• is related to: LU-10755 sanity test 409 fails with 'Fail to cleanup the env!'
• is related to: LU-11418 hung threads on MDT and MDT won't umount
• is related to: LU-13099 ll_set_inode()) Can not initialize inode warning

Comments

[Lai Siyao]

All patches landed.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/33930/
Subject: LU-11681 lmv: disable remote file statahead
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 02b5a407081c88090f7237ef464a6f6c74139f67

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/33928/
Subject: LU-11681 lfsck: misc fixes in inserting shard
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4becec906d3a997a8ff4a8c10e39633ff69bd738

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/33927/
Subject: LU-11681 lfsck: misc fixes for dangling entry repair
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a6dafda245a4ee4c5a86f585914cf0dde87e420e

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/33926/
Subject: LU-11681 lfsck: read LMV from bottom object
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ef12cbccb0a5c50da0bb9c4c0bf17e51df5ca91b

[Andreas Dilger]

There are a bunch of failures in sanity.sh test_60g with "mkdir: cannot create directory '/mnt/lustre/d60g.sanity/new': Input/output error" (linked here from LU-11767 as a duplicate):
https://testing.whamcloud.com/sub_tests/0db5baa0-a326-11e9-8dbe-52540065bddc
https://testing.whamcloud.com/sub_tests/cdb41b3e-a462-11e9-8fc1-52540065bddc
https://testing.whamcloud.com/sub_tests/249487e4-a635-11e9-8fc1-52540065bddc
https://testing.whamcloud.com/sub_tests/d9edd372-a7cd-11e9-9e3d-52540065bddc
https://testing.whamcloud.com/sub_tests/5e57126a-a876-11e9-8dbe-52540065bddc
https://testing.whamcloud.com/sub_tests/7c0f940e-a9ba-11e9-bade-52540065bddc
https://testing.whamcloud.com/sub_tests/7c3941de-aa0d-11e9-bade-52540065bddc
https://testing.whamcloud.com/sub_tests/714d0a40-aa16-11e9-8fc1-52540065bddc

It looks like these patches need to be refreshed.

[Gerrit Updater]

James Nunez ([e-mail]) uploaded a new patch: https://review.whamcloud.com/34687
Subject: LU-11681 tests: stop running sanity 65i
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 430ec3ba2cc71fb99e8a2fc80adae4638f4f928f

[Minh Diep]

+1 on b2_12: https://testing.whamcloud.com/test_sets/503715b8-54bb-11e9-8e92-52540065bddc

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/33930
Subject: LU-11681 lmv: disable remote file statahead
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: cd9447f033d284d7bcbb0c60118af1605f740107

[Gerrit Updater]

Lai Siyao ([e-mail]) uploaded a new patch: https://review.whamcloud.com/33929
Subject: LU-11681 llite: clear lsm if dir lost its LMV
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d451f4ee49eec34f56a5bebc1795bf1f9123833e