https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-15604 Lustre <p>This issue was created by maloo for Chris Horn &lt;chris.horn@hpe.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.whamcloud.com/test_sets/b3098e8b-70ca-4980-a955-159ee6537597" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/b3098e8b-70ca-4980-a955-159ee6537597</a></p> <p>test_226 failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>failed to unload modules </pre> </div></div> <p>This is a new test case added by the patch. I don't see anything obviously wrong, so I'm not sure why it failed in this way. I repeated the test 100 times in my VM environment and I did not see this issue.</p> <p>VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV<br/> sanity-lnet test_226 - failed to unload modules</p> LU-15604

sanity-lnet test_226: failed to unload modules

Bug Minor Closed Won't Fix Chris Horn Maloo Mon, 28 Feb 2022 16:58:20 +0000 Thu, 24 Mar 2022 15:49:55 +0000 Thu, 24 Mar 2022 15:49:55 +0000 0 2 <p>I was testing <a href="https://review.whamcloud.com/#/c/46727/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/46727/</a> by starting/stopping LNet in a loop while pinging the test node from 3 other peers:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@s-lmo-gaz38a ~]# while true; do for i in {1..20}; do lctl ping 172.18.2.6@o2ib10; done; echo sleep 5; sleep 5; done ... [root@s-lmo-gaz38b ~]# while true; do for i in {1..2}; do lnetctl discover --force 172.18.2.6@o2ib10; done; lnetctl peer del --prim 172.18.2.6@o2ib; echo sleep 5; sleep 5; done ... cassini-hosta:~ # while true; do for i in {1..20}; do lnetctl discover --force 172.18.2.6@o2ib; done; lnetctl peer del --prim 172.18.2.6@o2ib; echo sleep 5; sleep 5; done ... cassini-hostb:~ # while true; do /bin/start.sh2 ; lustre_rmmod ; echo sleep 5; sleep 5; done sleep 5 sleep 5 sleep 5 sleep 5 sleep 5 sleep 5 ... </pre> </div></div> <p>After a while, I opened some additional terminals on the test node and ran some lctl and lnetctl commands in a loop:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>cassini-hostb:~ # while true; do lctl list_nids 2&gt;/dev/null; done ... cassini-hostb:~ # while true; do lnetctl peer show 2&gt;/dev/null; done ... cassini-hostb:~ # while true; do lnetctl net show 2&gt;/dev/null; done ... </pre> </div></div> <p>At that point, I started to see some rmmod failures like in this ticket:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>sleep 5 sleep 5 rmmod: ERROR: Module libcfs is in use </pre> </div></div> <p>So I think that this test failure is most likely just from some other lctl or lnetctl process that is running at same time as rmmod. We're executing unload_modules_local via do_rpc_nodes() which is going to invoke "lctl mark" on all the test nodes. This is most likely what is causing the occasional rmmod failure. So it seems that it is not safe to call unload_modules_local() via do_rpc_nodes().</p> <p>The tests were modified to call lustre_rmmod via do_nodes() instead of the using unload_modules_locals() via do_rpc_nodes(). With that change, the test now passes consistently. <a href="https://testing.whamcloud.com/test_sessions/81932feb-6028-40fb-868d-f6a21485811c" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sessions/81932feb-6028-40fb-868d-f6a21485811c</a> 100/100</p> Development Rank 1|i02jlr: Rank (Obsolete) 9223372036854775807 Severity