https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5721 Lustre <p>Recently, Maloo tests hit some sanity-lfsck test_18f failures because of unexpected layout LFSCK status:</p> <p>sanity-lfsck test 18f: Skip the failed OST(s) when handle orphan OST-objects == 12:43:28 (1407761008)</p> <p><a href="https://testing.hpdd.intel.com/test_sets/29aaa042-4a50-11e4-880b-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/29aaa042-4a50-11e4-880b-5254006e85c2</a><br/> 10-02 11390/29 5d5f83237aba013d9bfaee0bc101fa403008e528 <a href="https://jira.whamcloud.com/browse/LU-5516" title="LFSCK 3: repair the lost name entry" class="issue-link" data-issue-key="LU-5516"><del>LU-5516</del></a> lfsck: repair the lost name entry<br/> 11384/29 e9299a16ed3b8402ac951994a04aed0876c3a365 <a href="https://jira.whamcloud.com/browse/LU-5515" title="LFSCK 3: repair bad file type in name entry" class="issue-link" data-issue-key="LU-5515"><del>LU-5515</del></a> lfsck: repair bad file type in name entry<br/> 11383/28 201b1e2570a43e0929454e46c2a8ef90df67d304 <a href="https://jira.whamcloud.com/browse/LU-5513" title="LFSCK 3: repair multiple referenced name entry" class="issue-link" data-issue-key="LU-5513"><del>LU-5513</del></a> lfsck: repair multiple referenced name entry<br/> Error: '(8) MDS1 is not the expected 'completed''</p> <p><a href="https://testing.hpdd.intel.com/test_sets/a4396266-3b47-11e4-a78f-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/a4396266-3b47-11e4-a78f-5254006e85c2</a><br/> 09-13 11384/20 f3b4b0d9c8e12d79ac7c76ea25685b8b78afb410 <a href="https://jira.whamcloud.com/browse/LU-5515" title="LFSCK 3: repair bad file type in name entry" class="issue-link" data-issue-key="LU-5515"><del>LU-5515</del></a> lfsck: repair bad file type in name entry<br/> 11383/20 <a href="https://jira.whamcloud.com/browse/LU-5513" title="LFSCK 3: repair multiple referenced name entry" class="issue-link" data-issue-key="LU-5513"><del>LU-5513</del></a> lfsck: repair multiple referenced name entry<br/> Error: '(8) MDS1 is not the expected 'completed''</p> <p><a href="https://testing.hpdd.intel.com/test_sets/2ed4fa44-3a52-11e4-b82a-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/2ed4fa44-3a52-11e4-b82a-5254006e85c2</a><br/> 09-11 11485/10 1b05a4f6d0e3b09bf50cf483a2b587f7f67242ac <a href="https://jira.whamcloud.com/browse/LU-5509" title="LFSCK 3: get PFID from linkEA for remote directory on ldiskfs" class="issue-link" data-issue-key="LU-5509"><del>LU-5509</del></a> osd: get PFID from linkEA for remote dir on ldiskfs<br/> 11382/10 091db2912495f692e38e2c20f40452e4925702af <a href="https://jira.whamcloud.com/browse/LU-5508" title="LFSCK 3: RPC adjustment for remote transaction" class="issue-link" data-issue-key="LU-5508"><del>LU-5508</del></a> osp: RPC adjustment for remote transaction<br/> 10996/18 ab05d3ba9c21125fc8194efb06545c358d962f3f <a href="https://jira.whamcloud.com/browse/LU-5506" title="LFSCK 3: Skip orphan objects handling for failed servers" class="issue-link" data-issue-key="LU-5506"><del>LU-5506</del></a> lfsck: skip orphan OST-object handling for failed OSTs (Merged commit).<br/> Error: '(4) MDS4 is not the expected 'completed''</p> <p><a href="https://testing.hpdd.intel.com/test_sets/aa055074-3543-11e4-9daf-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/aa055074-3543-11e4-9daf-5254006e85c2</a><br/> 09-05 11384/14 54ac922c31c41a0752367587e2692ef3747012bf <a href="https://jira.whamcloud.com/browse/LU-5515" title="LFSCK 3: repair bad file type in name entry" class="issue-link" data-issue-key="LU-5515"><del>LU-5515</del></a> lfsck: repair bad file type in name entry<br/> 11383/14 9cd74486d1e98ff6492c11e5f97fc873087ed7d4 <a href="https://jira.whamcloud.com/browse/LU-5513" title="LFSCK 3: repair multiple referenced name entry" class="issue-link" data-issue-key="LU-5513"><del>LU-5513</del></a> lfsck: repair multiple referenced name entry<br/> Error: '(2) MDS4 is not the expected 'partial''</p> <p><a href="https://testing.hpdd.intel.com/test_sets/2c7eb87c-2196-11e4-8700-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/2c7eb87c-2196-11e4-8700-5254006e85c2</a><br/> 08-11 11391/1 <a href="https://jira.whamcloud.com/browse/LU-5516" title="LFSCK 3: repair the lost name entry" class="issue-link" data-issue-key="LU-5516"><del>LU-5516</del></a> lfsck: repair orphan parent MDT-object<br/> 11390/1 5901fac8b083883dba6e396f73097b11a638659b <a href="https://jira.whamcloud.com/browse/LU-4788" title="LFSCK 3: MDT-MDT consistency verification" class="issue-link" data-issue-key="LU-4788"><del>LU-4788</del></a> lfsck: repair the lost name entry<br/> 11384/2 b7f3359b8cd82d208ff427febce92b1202e50a72 <a href="https://jira.whamcloud.com/browse/LU-5515" title="LFSCK 3: repair bad file type in name entry" class="issue-link" data-issue-key="LU-5515"><del>LU-5515</del></a> lfsck: repair bad file type in name entry<br/> 11383/3 290127554b47ed3871735d217e5c4c5b4d5fe365 <a href="https://jira.whamcloud.com/browse/LU-5513" title="LFSCK 3: repair multiple referenced name entry" class="issue-link" data-issue-key="LU-5513"><del>LU-5513</del></a> lfsck: repair multiple referenced name entry<br/> Error: '(2) MDS1 is not the expected 'partial''</p> <p><a href="https://testing.hpdd.intel.com/test_sets/876c3a80-2186-11e4-b153-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/876c3a80-2186-11e4-b153-5254006e85c2</a><br/> 08-11 11390/1 5901fac8b083883dba6e396f73097b11a638659b <a href="https://jira.whamcloud.com/browse/LU-4788" title="LFSCK 3: MDT-MDT consistency verification" class="issue-link" data-issue-key="LU-4788"><del>LU-4788</del></a> lfsck: repair the lost name entry<br/> 11384/2 b7f3359b8cd82d208ff427febce92b1202e50a72 <a href="https://jira.whamcloud.com/browse/LU-5515" title="LFSCK 3: repair bad file type in name entry" class="issue-link" data-issue-key="LU-5515"><del>LU-5515</del></a> lfsck: repair bad file type in name entry<br/> 11383/3 290127554b47ed3871735d217e5c4c5b4d5fe365 <a href="https://jira.whamcloud.com/browse/LU-5513" title="LFSCK 3: repair multiple referenced name entry" class="issue-link" data-issue-key="LU-5513"><del>LU-5513</del></a> lfsck: repair multiple referenced name entry<br/> Error: '(2) MDS1 is not the expected 'partial''</p> <p><a href="https://testing.hpdd.intel.com/test_sets/f38fb5c8-216f-11e4-bd4e-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/f38fb5c8-216f-11e4-bd4e-5254006e85c2</a><br/> 08-11 11383/3 290127554b47ed3871735d217e5c4c5b4d5fe365 <a href="https://jira.whamcloud.com/browse/LU-5513" title="LFSCK 3: repair multiple referenced name entry" class="issue-link" data-issue-key="LU-5513"><del>LU-5513</del></a> lfsck: repair multiple referenced name entry<br/> Error: '(2) MDS1 is not the expected 'partial''</p> LU-5721

sanity-lfsck test_18f failed for unexpected layout LFSCK status

Bug Critical Resolved Fixed nasf nasf Thu, 9 Oct 2014 13:41:10 +0000 Mon, 3 Nov 2014 01:19:35 +0000 Mon, 3 Nov 2014 01:19:35 +0000 Lustre 2.7.0 0 5 <p>Here are the other 9/16 failures in maloo when I checked yesterday:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>---- 11560 ---- https://testing.hpdd.intel.com/test_sets/da2714ee-49f8-11e4-92b1-5254006e85c2 10-02 11560 a168cdd6e093d1cb6fc551202d6a53aab5f87fc8/5 LU-5451 lod: improve weird FID handling 7e000f8fcad8ed9023f502ca63c47f3bdcac8a6b LU-5511 lfsck: repair unmatched parent-child pairs Error: '(6.1) Expect 1 fixed on mds1, but got: 0' https://testing.hpdd.intel.com/test_sets/b6842b5e-4a06-11e4-adcb-5254006e85c2 10-01 11560 a168cdd6e093d1cb6fc551202d6a53aab5f87fc8 LU-5451 lod: improve weird FID handling Error: '(6.1) Expect 1 fixed on mds1, but got: 0' https://testing.hpdd.intel.com/test_sets/47630d76-4560-11e4-8e96-5254006e85c2 09-26 11560 fa13c28d81ca917d1cfdfdefedb3a06845bb2386 LU-5451 lod: improve weird FID handling Error: '(6.1) Expect 1 fixed on mds1, but got: 0' ---- 10996 ---- https://testing.hpdd.intel.com/test_sets/883bc6b6-06be-11e4-8941-5254006e85c2 07-08 10996/4 2c4ffba41367e2cb850b2f7af1285641112c87fc LU-5506 lfsck: skip orphan OST-object handling for failed OSTs (Merged) Error: '(6) Expect 2 fixed on mds{2}, but got: 3' https://testing.hpdd.intel.com/test_sets/aef7bb80-0645-11e4-8bf0-5254006e85c2 07-08 10996/3 e5d34ebcb64476bc9228551d68f08f0de4ae2944 LU-5506 lfsck: skip orphan OST-object handling for failed OSTs Error: '(3) OST{1} Expect 'partial', but got 'scanning-phase2'' https://testing.hpdd.intel.com/test_sets/6c7a8fea-0617-11e4-be6f-5254006e85c2 07-07 10996/2 e5d34ebcb64476bc9228551d68f08f0de4ae2944 LU-5506 lfsck: skip orphan OST-object handling for failed OSTs Error: '(6) Expect 2 fixed on mds{2}, but got: 3' ---- full ---- https://testing.hpdd.intel.com/test_sets/64eb3bde-4dfd-11e4-8fdd-5254006e85c2 10-06 full Error: '(2) MDS1 is not the expected 'partial'' client 6039fc8fd47ffd73a31b073687f32cac0a35a8aa v2_6_53_0-12-g6039fc8 server 73ea776053d99f74a9f5679fe55ec5d9461b8a89 v2_6_0_0 https://testing.hpdd.intel.com/test_sets/44f280d4-4d45-11e4-857c-5254006e85c2 10-05 full Error: '(2) MDS1 is not the expected 'partial'' client 0b4b33592c09d37c0132d39c7823db78a3efcb3c v2_6_53_0-8-g0b4b335 server 73ea776053d99f74a9f5679fe55ec5d9461b8a89 v2_6_0_0 ---- 9383 ---- https://testing.hpdd.intel.com/test_sets/4b097264-48ce-11e4-b83b-5254006e85c2 09-30 9383 LU-4665 utils: lfs setstripe to specify OSTs Error: '(3) Fail to repair unmatched pair: 0' DUE TO REGRESSION IN PATCH. </pre> </div></div> <p>What is the impact of this bug? It isn't at all clear from the description whether this is just causing test failures, or if it is actually a bug that would cause problems for users?</p> <p>There are two sub-failures under this ticket:</p> <p>1) During the first part of test test, we inject error stub to similar the case of some OST failed to respond some LFSCK request, then the LFSCK on the MDT should skip orphan OST-object handling for this OST and mark the LFSCK status as "partial", but because of some unknown reason, such injection did not cause the "partial" status. The potential impact is that the LFSCK may handle some orphan OST-objects unexpectedly. But because the OST-object usually contains its parent MDT-object's FID information, such unexpected LFSCK behaviour is harmless for most of cases.</p> <p>2) During the second part of the test, we clear former injected error stub, then the LFSCK should go smoothly and handle all objects. The final LFSCK status should be "completed", but because of some unknown reason, the LFSCK failed to verify some OST-object(s), then the final LFSCK status was "partial". This failure may cause some orphan OST-objects cannot be handled.</p> <p>1) The failure about "Error: '(2) MDS1 is not the expected 'partial'' in the old version tests:<br/> <a href="https://testing.hpdd.intel.com/test_sets/f38fb5c8-216f-11e4-bd4e-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/f38fb5c8-216f-11e4-bd4e-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/876c3a80-2186-11e4-b153-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/876c3a80-2186-11e4-b153-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/2c7eb87c-2196-11e4-8700-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/2c7eb87c-2196-11e4-8700-5254006e85c2</a></p> <p>They failed because the injected failure stub has not been triggered. Such issue has already been resolved by subsequent versions and has been landed to master.</p> <p>2) The failure about "Error: '(2) MDS4 is not the expected 'partial'' in the tests:<br/> <a href="https://testing.hpdd.intel.com/test_sets/aa055074-3543-11e4-9daf-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/aa055074-3543-11e4-9daf-5254006e85c2</a></p> <p>It is another failure instance <a href="https://jira.whamcloud.com/browse/LU-5301" title="Test failure sanity-lfsck test_17: (3) unexpected status" class="issue-link" data-issue-key="LU-5301"><del>LU-5301</del></a>.</p> <p>3) The failure about "Error: '(4) MDS4 is not the expected 'completed'' in the tests:<br/> <a href="https://testing.hpdd.intel.com/test_sets/2ed4fa44-3a52-11e4-b82a-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/2ed4fa44-3a52-11e4-b82a-5254006e85c2</a></p> <p>It is another failure instance <a href="https://jira.whamcloud.com/browse/LU-5301" title="Test failure sanity-lfsck test_17: (3) unexpected status" class="issue-link" data-issue-key="LU-5301"><del>LU-5301</del></a>.</p> <p>4) The failure about "Error: '(8) MDS1 is not the expected 'completed'' in the tests:<br/> <a href="https://testing.hpdd.intel.com/test_sets/a4396266-3b47-11e4-a78f-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/a4396266-3b47-11e4-a78f-5254006e85c2</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/29aaa042-4a50-11e4-880b-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/29aaa042-4a50-11e4-880b-5254006e85c2</a></p> <p>The LFSCK on the OST got some abnormal status when queried the LFSCK status from the MDT, then the LFSCK on the OST thought that the LFSCK on the MDT hit some unexpected trouble and marked them as exit in advance, and then the subsequent orphan MDT-object handling has been skipped. This issue will be fixed via the patch: <a href="http://review.whamcloud.com/#/c/11516/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/11516/</a></p> <p>another seen in master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/464f8b14-5fcb-11e4-9a8e-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/464f8b14-5fcb-11e4-9a8e-5254006e85c2</a></p> <p>The patch <a href="http://review.whamcloud.com/#/c/11516/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/11516/</a> has been landed to master at Oct.30th, other related fixes will be landed to master via <a href="https://jira.whamcloud.com/browse/LU-5301" title="Test failure sanity-lfsck test_17: (3) unexpected status" class="issue-link" data-issue-key="LU-5301"><del>LU-5301</del></a>/<a href="https://jira.whamcloud.com/browse/LU-5731" title="sanity-lfsck test_18a: " class="issue-link" data-issue-key="LU-5731"><del>LU-5731</del></a>.</p> Development Rank 1|hzwy67: Rank (Obsolete) 16053 Severity