[LU-10755] sanity test 409 fails with 'Fail to cleanup the env!'

• Type: Bug
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.11.0, Lustre 2.13.0, Lustre 2.12.2
• Labels: dne zfs
• Severity: 3
• Assignee: WC Triage
• Reporter: James Nunez (Inactive)

Description

sanity test_409 fail to clean up the tes directory because the “Directory is not empty”. From the suite_log, we see

== sanity test 409: Large amount of cross-MDTs hard links on the same file =========================== 08:11:58 (1511338318)
rm: cannot remove '/mnt/lustre/d162c.sanity.remote/0/1/2': Directory not empty
 sanity test_409: @@@@@@ FAIL: Fail to cleanup the env!

The test suite also fails to clean up the test directory for the same reason

== sanity test complete, duration 9662 sec =========================================================== 08:15:06 (1511338506)
sanity: FAIL: test_409 Fail to cleanup the env!
rm: cannot remove '/mnt/lustre/d162c.sanity.remote/0/1/2': Directory not empty
 sanity : @@@@@@ FAIL: remove sub-test dirs failed 

In some of these failures, we see files from a different sanity test that cannot be removed

== sanity test 409: Large amount of cross-MDTs hard links on the same file =========================== 13:58:42 (1518875922)
rm: cannot remove '/mnt/lustre/d140.sanity/1/2/3/4': Directory not empty
 sanity test_409: @@@@@@ FAIL: Fail to cleanup the env! 

In all of these cases, the test that left a directory and files that cannot be removed did not fail … at least, they did not fail with a call to error().

Test suites after sanity will also fail trying to remove the contents of the mount point with this error

sanity test 409 started failing with this error starting on 2017-11-22.

Some logs for these failures are at
https://testing.hpdd.intel.com/test_sets/458fbce6-d05a-11e7-a066-52540065bddc
https://testing.hpdd.intel.com/test_sets/3e035a3c-f828-11e7-bd00-52540065bddc
https://testing.hpdd.intel.com/test_sets/5cfd75f4-13ed-11e8-a7cd-52540065bddc

Issue Links

• is related to: LU-9883 sanity test 407 rm: cannot remove '/mnt/lustre/d407.sanity.1': Directory not empty
• is related to: LU-11681 sanity test 65i fails with 'find /mnt/lustre failed'