https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5241 Lustre <p>sanity-lfsck test 0 failed as follows:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Started LFSCK on the device lustre-MDT0000: namespace. CMD: shadow-4vm4 /usr/sbin/lctl get_param -n mdd.lustre-MDT0000.lfsck_namespace name: lfsck_namespace magic: 0xa0629d03 version: 2 status: scanning-phase1 flags: param: time_since_last_completed: N/A time_since_latest_start: 1 seconds time_since_last_checkpoint: N/A latest_start_position: 13, N/A, N/A last_checkpoint_position: N/A, N/A, N/A first_failure_position: N/A, N/A, N/A checked_phase1: 0 checked_phase2: 0 updated_phase1: 0 updated_phase2: 0 failed_phase1: 0 failed_phase2: 0 dirs: 0 M-linked: 0 nlinks_repaired: 0 lost_found: 0 success_count: 0 run_time_phase1: 0 seconds run_time_phase2: 0 seconds average_speed_phase1: 0 items/sec average_speed_phase2: N/A real-time_speed_phase1: 0 items/sec real-time_speed_phase2: N/A current_position: 12, N/A, N/A CMD: shadow-4vm4 /usr/sbin/lctl get_param -n mdd.lustre-MDT0000.lfsck_namespace CMD: shadow-4vm4 /usr/sbin/lctl lfsck_stop -M lustre-MDT0000 Stopped LFSCK on the device lustre-MDT0000. CMD: shadow-4vm4 /usr/sbin/lctl get_param -n mdd.lustre-MDT0000.lfsck_namespace CMD: shadow-4vm4 /usr/sbin/lctl lfsck_start -M lustre-MDT0000 -t namespace Started LFSCK on the device lustre-MDT0000: namespace. CMD: shadow-4vm4 /usr/sbin/lctl get_param -n mdd.lustre-MDT0000.lfsck_namespace CMD: shadow-4vm4 /usr/sbin/lctl set_param fail_loc=0 fail_loc=0 CMD: shadow-4vm4 /usr/sbin/lctl set_param fail_val=0 fail_val=0 CMD: shadow-4vm4 /usr/sbin/lctl get_param -n mdd.lustre-MDT0000.lfsck_namespace sanity-lfsck test_0: @@@@@@ FAIL: (9) Expect 'completed', but got 'scanning-phase1' </pre> </div></div> <p>Maloo report: <a href="https://maloo.whamcloud.com/test_sets/8d110b74-f903-11e3-9283-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/8d110b74-f903-11e3-9283-52540035b04c</a></p> Lustre client build: <a href="https://build.hpdd.intel.com/job/lustre-b2_4/73/">https://build.hpdd.intel.com/job/lustre-b2_4/73/</a> (2.4.3)<br/> Lustre server build: <a href="https://build.hpdd.intel.com/job/lustre-b2_5/74/">https://build.hpdd.intel.com/job/lustre-b2_5/74/</a> (2.5.2 RC2)<br/> LU-5241

2.4.3<->2.5.2 interop: sanity-lfsck test_0: FAIL: (9) Expect 'completed', but got 'scanning-phase1'

Bug Critical Resolved Fixed Emoly Liu Jian Yu mn4 Sat, 21 Jun 2014 15:37:11 +0000 Sun, 14 Jun 2015 13:36:18 +0000 Thu, 4 Dec 2014 22:39:35 +0000 Lustre 2.5.2 Lustre 2.4.3 Lustre 2.5.4 0 6 <p>Hi Nasf,</p> <p>Could you please take a look at the failure to see whether this is an issue on Lustre b2_5 side? Thanks.</p> <p>The failure is related with the following test scripts:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> do_facet $SINGLEMDS $LCTL set_param fail_loc=0 do_facet $SINGLEMDS $LCTL set_param fail_val=0 sleep 3 STATUS=$($SHOW_NAMESPACE | awk <span class="code-quote">'/^status/ { print $2 }'</span>) [ <span class="code-quote">"$STATUS"</span> == <span class="code-quote">"completed"</span> ] || error <span class="code-quote">"(9) Expect <span class="code-quote">'completed'</span>, but got <span class="code-quote">'$STATUS'</span>"</span> </pre> </div></div> <p>From the test log, I cannot find any abnormal cases to indicate potential Lustre bugs. Instead, I suspect that it is related with the "sleep 3". Because "sleep 3" is an average estimated time that the LFSCK can finish the scanning, but such estimation may be affected by kinds of facts, such as VM scheduler trouble. We have improved the test scripts in the master as following:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> do_facet $SINGLEMDS $LCTL set_param fail_loc=0 fail_val=0 wait_update_facet $SINGLEMDS "$LCTL get_param -n \ mdd.${MDT_DEV}.lfsck_namespace | awk <span class="code-quote">'/^status/ { print \\\$2 }'</span><span class="code-quote">" "</span>completed" 6 || { $SHOW_NAMESPACE error <span class="code-quote">"(9) unexpected status"</span> } </pre> </div></div> <p>So if possible, we should back-port the patch <a href="http://review.whamcloud.com/9704" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9704</a> to b2_5 and b2_4. Such patch improved the sanity-scrub/sanity-lfsck test scripts.</p> <p>Thanks Fanyong. Emoly, could you please make the appropriate change to the test for b2_5 and b2_4?</p> <p>The backported patch to b2_5 is here: <a href="http://review.whamcloud.com/#/c/10818/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/10818/</a><br/> The backported patch to b2_4 is here: <a href="http://review.whamcloud.com/#/c/10892/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/10892/</a></p> <p>This problem is being blocked by <a href="https://jira.whamcloud.com/browse/LU-5248" title="Test failure on sanity-lfsck.sh, subtest test_4: ls: reading directory /mnt/lustre/d4.sanity-lfsck: Input/output error" class="issue-link" data-issue-key="LU-5248"><del>LU-5248</del></a>. We should land that fix first.</p> <p>The back-ported patch for Lustre b2_5 branch <a href="http://review.whamcloud.com/10818" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10818</a> was updated to depend on <a href="http://review.whamcloud.com/11006" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/11006</a>, which is the patch for <a href="https://jira.whamcloud.com/browse/LU-5248" title="Test failure on sanity-lfsck.sh, subtest test_4: ls: reading directory /mnt/lustre/d4.sanity-lfsck: Input/output error" class="issue-link" data-issue-key="LU-5248"><del>LU-5248</del></a>.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/10818/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/10818/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5241" title="2.4.3&lt;-&gt;2.5.2 interop: sanity-lfsck test_0: FAIL: (9) Expect &#39;completed&#39;, but got &#39;scanning-phase1&#39;" class="issue-link" data-issue-key="LU-5241"><del>LU-5241</del></a> tests: speed up sanity-lfsck and sanity-scrub tests<br/> Project: fs/lustre-release<br/> Branch: b2_5<br/> Current Patch Set: <br/> Commit: 2ab8b98ea5dafbce59043e5d8477e794197116a0</p> <p>Landed for 2.5.4</p> Blocker LU-5248 Development Rank 1|hzwpnb: Rank (Obsolete) 14614 Severity