https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3143 Lustre <p>The recovery-small test 29a failed as follows:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>CMD: client-32vm3 lctl get_param -n at_min error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match error: list_param: /proc/{fs,sys}/{lnet,lustre}/osc/lustre-OST0000-osc-MDT0000/ost_server_uuid: Found no match can't get osc.lustre-OST0000-osc-MDT0000.ost_server_uuid by list_param in 40 secs Go with osc.lustre-OST0000-osc-MDT0000.ost_server_uuid directly CMD: client-32vm3 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/lib64/openmpi/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin: NAME=autotest_config sh rpc.sh wait_import_state FULL osc.lustre-OST0000-osc-MDT0000.ost_server_uuid 40 client-32vm3: rpc : @@@@@@ FAIL: can't put import for osc.lustre-OST0000-osc-MDT0000.ost_server_uuid into FULL state after 40 sec, have </pre> </div></div> <p>Maloo report: <a href="https://maloo.whamcloud.com/test_sets/740b8974-a1bb-11e2-bdac-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/740b8974-a1bb-11e2-bdac-52540035b04c</a></p> <br/> Lustre Branch: master<br/> Lustre Build: <a href="http://build.whamcloud.com/job/lustre-master/1381/">http://build.whamcloud.com/job/lustre-master/1381/</a><br/> Distro/Arch: RHEL6.3/x86_64<br/> Test Group: failover<br/> FAILURE_MODE=HARD<br/> LU-3143

recovery-small test_29a: FAIL: import is not in FULL state

Bug Blocker Resolved Fixed Hongchao Zhang Jian Yu LB Wed, 10 Apr 2013 15:21:11 +0000 Wed, 26 Jun 2013 10:22:47 +0000 Thu, 25 Apr 2013 17:06:52 +0000 Lustre 2.4.0 Lustre 2.4.0 0 6 <p>Can we get MDS conman output fo this test? it almost looks like MDS failed to start after failover, but there are not mds locks in the report at all so hard to tell the real cause.</p> <p>Hongchao</p> <p>Could you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>It seems "$(facet_host $facet)" in wait_osc_import_state() should be changed to "$(facet_active_host $facet)".</p> <p>the patch is tracked at <a href="http://review.whamcloud.com/#change,6036" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,6036</a></p> <p>Landed for 2.4</p> <blockquote><p>the patch is tracked at <a href="http://review.whamcloud.com/#change,6036" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,6036</a></p></blockquote> <p>The Maloo report for patch set 2 showed that the recovery-small test 29a still failed.</p> <p>there is still one more problem in the test script, the facet name of MDT is different when calling wait_osc_import_state</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">test_29a() { # bug 22273 - error adding <span class="code-keyword">new</span> clients #define OBD_FAIL_TGT_CLIENT_ADD 0x711 do_facet $SINGLEMDS <span class="code-quote">"lctl set_param fail_loc=0x80000711"</span> # fail abort so client will be <span class="code-keyword">new</span> again fail_abort $SINGLEMDS client_up || error <span class="code-quote">"reconnect failed"</span> wait_osc_import_state mds ost FULL &lt;--- here, should use $SINGLEMDS (mds1) <span class="code-keyword">return</span> 0 } </pre> </div></div> <p>there are some other places (mainly in conf-sanity.sh) with the same problem needed to fix, otherwise it could fail in failover mode with FAILURE_MODE=HARD</p> <p>the patch is tracked at <a href="http://review.whamcloud.com/#change,6100" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,6100</a></p> <p>Landed for 2.4</p> <p>Just got a new occurrence in <a href="https://maloo.whamcloud.com/test_sets/e79d863e-de3e-11e2-b04c-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/e79d863e-de3e-11e2-b04c-52540035b04c</a>.</p> Development Rank 1|hzvnjr: Rank (Obsolete) 7629 Severity