https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-14153 Lustre <p>This issue was created by maloo for Andreas Dilger &lt;adilger@whamcloud.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.whamcloud.com/test_sets/4a0c244b-cc64-455a-9d41-a8cf9790407c" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/4a0c244b-cc64-455a-9d41-a8cf9790407c</a></p> <p>test_280 failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>== sanity test 280: Race between MGS umount and client llog processing ===== 10.9.6.22@tcp:/lustre /mnt/lustre lustre rw,seclabel,flock,user_xattr,lazystatfs,noencrypt 0 0 CMD: trevis-45vm1.trevis.whamcloud.com grep -c /mnt/lustre' ' /proc/mounts Stopping client trevis-45vm1.trevis.whamcloud.com /mnt/lustre (opts:) CMD: trevis-45vm1.trevis.whamcloud.com lsof -t /mnt/lustre CMD: trevis-45vm1.trevis.whamcloud.com umount /mnt/lustre 2&gt;&amp;1 CMD: trevis-25vm1.trevis.whamcloud.com mount -t lustre -o user_xattr,flock,skpath=/tmp/test-framework-keys trevis-25vm4@tcp:/lustre /mnt/lustre : Starting client: trevis-25vm1.trevis.whamcloud.com: -o user_xattr,flock,skpath=/tmp/test-framework-keys trevis-25vm4@tcp:/lustre /mnt/lustre CMD: trevis-25vm1.trevis.whamcloud.com mkdir -p /mnt/lustre CMD: trevis-25vm1.trevis.whamcloud.com mount -t lustre -o user_xattr,flock,skpath=/tmp/test-framework-keys trevis-25vm4@tcp:/lustre /mnt/lustre mount.lustre: according to /etc/mtab trevis-25vm4@tcp:/lustre is already mounted on /mnt/lustre sanity test_280: @@@@@@ FAIL: mount client failed </pre> </div></div> <p>It looks like this is failing intermittently since 2020-07-30 (about 35 times over 4 months) for <tt>review-dne-ssk</tt> and <tt>review-dne-selinux-ssk</tt> sessions. Note that it does not happen for <tt>review-dne-selinux</tt> sessions, and the few failures on other test sessions look like they were related to many previous tests failing also.</p> <p>It may just be a test script issue (e.g. SSK is causing the client mount to be slower, and the race that the subtest is trying to trigger is happening differently as a result).</p> <p>VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV<br/> sanity test_280 - mount client failed</p> LU-14153

sanity test_280: "mount client failed" on review-dne-ssk review-dne-selinux-ssk

Bug Minor Open Unresolved WC Triage Maloo SSK Fri, 27 Nov 2020 05:23:58 +0000 Mon, 25 Jan 2021 22:25:25 +0000 Lustre 2.14.0 0 4 <p>Comparing a passing and failing test run, it is clear that the <b>failing</b> test run has an additional "<tt>Starting client:</tt>" step, which leads to the "<tt>already mounted</tt>" error, because the <b>passing</b> test run shows that the client already has a mounted filesystem when <tt>mount_client()</tt> is called, so doesn't even try to mount it again:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>diff -u /tmp/passed /tmp/failed --- /tmp/passed 2020-11-26 22:38:25.000000000 -0700 +++ /tmp/failed 2020-11-26 22:38:26.000000000 -0700 @@ -33,6 +33,9 @@ pdsh@trevis-10vm1: trevis-10vm4: ssh exited with exit code 1 CMD: trevis-10vm4 e2label /dev/mapper/mds1_flakey 2&gt;/dev/null Started lustre-MDT0000 -client@tcp:/lustre /mnt/lustre lustre rw,seclabel,flock,user_xattr,lazystatfs,noencrypt 0 0 -Resetting fail_loc on all nodes...CMD: trevis-10vm1.trevis.whamcloud.com,trevis-10vm2,trevis-10vm3,trevis-10vm4,trevis-10vm5 lctl set_param -n fail_loc=0 fail_val=0 2&gt;/dev/null -done. +Starting client: trevis-10vm1.trevis.whamcloud.com: -o user_xattr,flock,skpath=/tmp/test-framework-keys trevis-10vm4@tcp:/lustre /mnt/lustre +CMD: trevis-10vm1.trevis.whamcloud.com mkdir -p /mnt/lustre +CMD: trevis-10vm1.trevis.whamcloud.com mount -t lustre -o user_xattr,flock,skpath=/tmp/test-framework-keys trevis-10vm4@tcp:/lustre /mnt/lustre +mount.lustre: according to /etc/mtab trevis-10vm4@tcp:/lustre is already mounted on /mnt/lustre + sanity test_280: @@@@@@ FAIL: mount client failed </pre> </div></div> <p>This definitely seems like a race in the test, since the failed test doesn't detect the mount in "<tt>mount_client()</tt>" but finds it later when <tt>zconf_mount()</tt> tries to mount.</p> <p>+1 <a href="https://testing.whamcloud.com/test_sets/2f812d63-e312-49fe-a7e7-67df1e801949" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/2f812d63-e312-49fe-a7e7-67df1e801949</a></p> <p>+1 on master: <a href="https://testing.whamcloud.com/test_sets/32b28fef-bddf-422b-9c73-e677bce9cc50" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/32b28fef-bddf-422b-9c73-e677bce9cc50</a></p> Development Rank 1|i01fvr: Rank (Obsolete) 9223372036854775807 Severity