[LU-13002] sanity-lnet stops all test suites from running Created: 22/Nov/19  Updated: 13/Jun/22  Resolved: 20/Dec/19

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.13.0, Lustre 2.14.0
Fix Version/s: Lustre 2.14.0

Type: Bug Priority: Minor
Reporter: James Nunez (Inactive) Assignee: James Nunez (Inactive)
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Related
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

sanity-lnet has some issues cleaning up at the end of the test suite that put Lustre (VMs?) into a bad state and does not allow other test suites to run. Looking at a recent test session failure at https://testing.whamcloud.com/test_sessions/1ab7866a-d3b5-4a48-b22b-953a35bf950a, we can see that sanity-lnet runs to completion, but all other test suites fail with

CMD: trevis-47vm8 loop_dev=\$(losetup -j /dev/mapper/mds1_flakey | cut -d : -f 1);
			 if [[ -z \$loop_dev ]]; then
				loop_dev=\$(losetup -f);
				losetup \$loop_dev /dev/mapper/mds1_flakey || loop_dev=;
			 fi;
			 echo -n \$loop_dev
trevis-47vm8: losetup: /dev/mapper/mds1_flakey: failed to set up loop device: No such file or directory

The last thing we see in the sanity-lnet suite_log is

CMD: trevis-47vm5.trevis.whamcloud.com lsmod | grep lnet > /dev/null &&
lctl dl | grep ' ST ' || true
opening /dev/obd failed: No such file or directory
hint: the kernel modules may not be loaded
modules unloaded.


 Comments   
Comment by James Nunez (Inactive) [ 23/Nov/19 ]

Chris - Any thoughts on this issue?

Comment by Chris Horn [ 24/Nov/19 ]

The test-suite leaves the system without any lustre running and all modules unloaded. Perhaps the tests are written such that they assume the filesystem is set up/running?

Comment by Gerrit Updater [ 25/Nov/19 ]

James Nunez (jnunez@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36849
Subject: LU-13002 tests: change clean up in sanity-lnet
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ae6fc5d6da4f146956d09947f8d6ce7ada34d9bb

Comment by Gerrit Updater [ 20/Dec/19 ]

Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36849/
Subject: LU-13002 tests: change clean up in sanity-lnet
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 43540e83b692dddb89055afbc393b79532659ad2

Comment by Peter Jones [ 20/Dec/19 ]

Landed for 2.14

Comment by Gerrit Updater [ 29/Nov/21 ]

"James Nunez <jnunez@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/45672
Subject: LU-13002 tests: change clean up in sanity-lnet
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 13db7cc42b41eb96db8aaa6e825cd23aac66808c

Generated at Sat Feb 10 02:57:31 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.