https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9820 Lustre <p>sanity-scrub test 9 fails with</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>sanity-scrub test_9: @@@@@@ FAIL: (9) Expected 'scanning' on mds3 </pre> </div></div> <p>From the test_log, the last thing we see is:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>CMD: trevis-8vm4 /usr/sbin/lctl lfsck_start -M lustre-MDT0000 -t scrub 8 -s 100 -r Started LFSCK on the device lustre-MDT0000: scrub CMD: trevis-8vm8 /usr/sbin/lctl lfsck_start -M lustre-MDT0001 -t scrub 8 -s 100 -r Started LFSCK on the device lustre-MDT0001: scrub CMD: trevis-8vm4 /usr/sbin/lctl lfsck_start -M lustre-MDT0002 -t scrub 8 -s 100 -r Started LFSCK on the device lustre-MDT0002: scrub CMD: trevis-8vm8 /usr/sbin/lctl lfsck_start -M lustre-MDT0003 -t scrub 8 -s 100 -r Started LFSCK on the device lustre-MDT0003: scrub CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0000.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0000.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm8 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0001.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm8 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0001.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0002.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0002.oi_scrub | awk '/^status/ { print \$2 }' Waiting 6 secs for update CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0002.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0002.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0002.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0002.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0002.oi_scrub | awk '/^status/ { print \$2 }' CMD: trevis-8vm4 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0002.oi_scrub | awk '/^status/ { print \$2 }' Update not seen after 6s: wanted 'scanning' got 'completed' sanity-scrub test_9: @@@@@@ FAIL: (9) Expected 'scanning' on mds3 </pre> </div></div> <p>There are no obvious errors in the console and debug logs.</p> <p>So far, this looks like it only impacts DNE Lustre configurations (reivew-dne-part-*)<br/> This test started to fail on June 2, 2017.</p> <p>Logs for this test failures are at:<br/> <a href="https://testing.hpdd.intel.com/test_sets/f934f42a-772a-11e7-8e04-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/f934f42a-772a-11e7-8e04-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/6112f68a-76f1-11e7-8db2-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/6112f68a-76f1-11e7-8db2-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/57e14740-7626-11e7-9a53-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/57e14740-7626-11e7-9a53-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/f429dfd0-733e-11e7-8d7d-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/f429dfd0-733e-11e7-8d7d-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/cd0041d8-6eac-11e7-a055-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/cd0041d8-6eac-11e7-a055-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/200eed24-6d32-11e7-a052-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/200eed24-6d32-11e7-a052-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/76ef4bb6-6841-11e7-a74b-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/76ef4bb6-6841-11e7-a74b-5254006e85c2</a></p> LU-9820

sanity-scrub test_9 fails with "(9) Expected 'scanning' on mds*"

Bug Minor Resolved Fixed Lai Siyao James Nunez Wed, 2 Aug 2017 15:49:43 +0000 Fri, 26 Feb 2021 14:36:27 +0000 Fri, 26 Feb 2021 14:36:27 +0000 Lustre 2.11.0 Lustre 2.10.3 Lustre 2.10.6 Lustre 2.10.7 Lustre 2.15.0 0 18 <p>more on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/b09ea26e-8cbb-11e7-b94a-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/b09ea26e-8cbb-11e7-b94a-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/c519fe96-951c-11e7-b89e-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/c519fe96-951c-11e7-b89e-5254006e85c2</a></p> <p>One more on master :<br/> <a href="https://testing.hpdd.intel.com/test_sessions/b7e37542-ad38-4029-8e4b-bc63b0d31e9d" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/b7e37542-ad38-4029-8e4b-bc63b0d31e9d</a></p> <p>another on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/d0440872-abcc-11e7-b78a-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/d0440872-abcc-11e7-b78a-5254006e85c2</a></p> <p>+1 on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/e1b33714-cc0f-11e7-8027-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/e1b33714-cc0f-11e7-8027-52540065bddc</a></p> <p><a href="https://testing.hpdd.intel.com/test_sessions/2b869baa-72d0-4873-9cb9-4dc7e80ae8bd" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/2b869baa-72d0-4873-9cb9-4dc7e80ae8bd</a></p> <p><a href="https://testing.hpdd.intel.com/test_sets/65acf810-df85-11e7-9840-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/65acf810-df85-11e7-9840-52540065bddc</a></p> <p>+2 on master branch:<br/> <a href="https://testing.hpdd.intel.com/test_sets/7d9da346-e658-11e7-9840-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/7d9da346-e658-11e7-9840-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/d848a0aa-ed35-11e7-a169-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/d848a0aa-ed35-11e7-a169-52540065bddc</a></p> <p>This looks like it fails because for DNE each MDT is waited for serially in <tt>sanity-scrub.sh::scrub_check_status()</tt>, thus if one of the middle MDTs takes a long time (test log shows "Waiting 6 secs for update") then the end ones may complete before being able to show up as "scanning".</p> <p>+1 on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/98f2733e-18ac-11e8-bd00-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/98f2733e-18ac-11e8-bd00-52540065bddc</a></p> <p>+1 on 2.10.3 for</p> <p>RHEL 7.4 Server/DNE/ldiskfs<br/> RHEL 7.4 Client</p> <p>build 96</p> <p><a href="https://testing.hpdd.intel.com/test_sets/6bba7824-3d84-11e8-960d-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/6bba7824-3d84-11e8-960d-52540065bddc</a></p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/40591" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40591</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9820" title="sanity-scrub test_9 fails with &quot;(9) Expected &#39;scanning&#39; on mds*&quot;" class="issue-link" data-issue-key="LU-9820"><del>LU-9820</del></a> osd-ldiskfs: OI scrub speed lmit fix<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 94ce8731c1ff9e95366b57c9ee919bd621af96a5</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/40591/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/40591/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9820" title="sanity-scrub test_9 fails with &quot;(9) Expected &#39;scanning&#39; on mds*&quot;" class="issue-link" data-issue-key="LU-9820"><del>LU-9820</del></a> osd-ldiskfs: OI scrub speed limit fix<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: fc8f1381698369565752f32815c4b38f9c1e8cdb</p> <p>Landed for 2.15</p> Related Development Rank 1|hzzhn3: Rank (Obsolete) 9223372036854775807 Severity