https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-5510 Lustre <p>sanity-scrub test 15 failed as follows:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Started LFSCK on the MDT device lustre-MDT0000. CMD: shadow-46vm3 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0000.oi_scrub CMD: shadow-46vm3 /usr/sbin/lctl get_param -n osd-ldiskfs.lustre-MDT0000.oi_scrub sanity-scrub test_15: @@@@@@ FAIL: (7) Expected 'inconsistent' on mds1, but got 'inconsistent,upgrade' </pre> </div></div> <p>Maloo report: <a href="https://testing.hpdd.intel.com/test_sets/6a6b507e-f8af-11e3-842c-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/6a6b507e-f8af-11e3-842c-5254006e85c2</a></p> Lustre client build: <a href="https://build.hpdd.intel.com/job/lustre-b2_5/80/">https://build.hpdd.intel.com/job/lustre-b2_5/80/</a><br/> Lustre server build: <a href="https://build.hpdd.intel.com/job/lustre-b2_4/73/">https://build.hpdd.intel.com/job/lustre-b2_4/73/</a> (2.4.3)<br/> Distro/Arch: RHEL6.5/x86_64 LU-5510

2.4.3<->2.5.3 interop: sanity-scrub test_15: FAIL: (7) Expected 'inconsistent' on mds1, but got 'inconsistent,upgrade'

Bug Minor Resolved Fixed nasf Jian Yu Wed, 20 Aug 2014 06:56:46 +0000 Wed, 11 Feb 2015 09:18:40 +0000 Sun, 8 Feb 2015 04:52:52 +0000 Lustre 2.7.0 Lustre 2.5.3 Lustre 2.7.0 0 6 <p>Hi Nasf,</p> <p>Is this a new interop failure related to the patches for <a href="https://jira.whamcloud.com/browse/LU-4058" title="Interop 2.4.0&lt;-&gt;2.5 failure on test suite sanity-scrub test_15" class="issue-link" data-issue-key="LU-4058"><del>LU-4058</del></a>?</p> <p>The failure did not occur in the interop testing between Lustre b2_5 build #83 client with Lustre 2.4.3 server.</p> <p>Lustre client build: <a href="https://build.hpdd.intel.com/job/lustre-b2_5/86/" class="external-link" target="_blank" rel="nofollow noopener">https://build.hpdd.intel.com/job/lustre-b2_5/86/</a> (2.5.3 RC1)<br/> Lustre server build: <a href="https://build.hpdd.intel.com/job/lustre-b2_4/73/" class="external-link" target="_blank" rel="nofollow noopener">https://build.hpdd.intel.com/job/lustre-b2_4/73/</a> (2.4.3)<br/> Distro/Arch: RHEL6.5/x86_64</p> <p>The failure occurred again: <a href="https://testing.hpdd.intel.com/test_sets/91e037a4-31f4-11e4-8d72-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/91e037a4-31f4-11e4-8d72-5254006e85c2</a></p> <p>Yujian,</p> <p>I do not think the issue is related with <a href="https://jira.whamcloud.com/browse/LU-4058" title="Interop 2.4.0&lt;-&gt;2.5 failure on test suite sanity-scrub test_15" class="issue-link" data-issue-key="LU-4058"><del>LU-4058</del></a>. It looks like that there were some objects without LMA EA, then the OI scrub regarded that the system was upgrading from Lustre-1.8. It may be not related with the interoperability, instead, it may be reproducible on Lustre-2.4.3 directly. Have you ever run the sanity-scrub on pure Lustre-2.4.3 without interoperating with Lustre-2.5.3?</p> <blockquote><p>Have you ever run the sanity-scrub on pure Lustre-2.4.3 without interoperating with Lustre-2.5.3?</p></blockquote> <p>Yes, and here are the full group test sessions on Lustre 2.4.3:<br/> <a href="https://testing.hpdd.intel.com/test_sessions/e75fa67e-ab62-11e3-a696-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/e75fa67e-ab62-11e3-a696-52540035b04c</a><br/> <a href="https://testing.hpdd.intel.com/test_sessions/3aaccf46-ac5d-11e3-81d7-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/3aaccf46-ac5d-11e3-81d7-52540035b04c</a><br/> <a href="https://testing.hpdd.intel.com/test_sessions/ffeb5ee0-abe4-11e3-bcad-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/ffeb5ee0-abe4-11e3-bcad-52540035b04c</a></p> <p>sanity-scrub test passed.</p> <p>However, I found the following failures on Lustre 2.4.2 full group test sessions:<br/> <a href="https://testing.hpdd.intel.com/test_sets/4f889346-6a5e-11e3-81c0-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/4f889346-6a5e-11e3-81c0-52540035b04c</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/fc56303c-6c58-11e3-92d0-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/fc56303c-6c58-11e3-92d0-52540035b04c</a></p> <p>It looks like the failure existed on Lustre b2_4 and occured sporadically.</p> <p>Fan Yong (fan.yong@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/13187" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13187</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5510" title="2.4.3&lt;-&gt;2.5.3 interop: sanity-scrub test_15: FAIL: (7) Expected &#39;inconsistent&#39; on mds1, but got &#39;inconsistent,upgrade&#39; " class="issue-link" data-issue-key="LU-5510"><del>LU-5510</del></a> scrub: ldiskfs_create_inode returns locked inode<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: cac7f444551d7d495d5dbc744c9114d1bdd77fbb</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/13187/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/13187/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-5510" title="2.4.3&lt;-&gt;2.5.3 interop: sanity-scrub test_15: FAIL: (7) Expected &#39;inconsistent&#39; on mds1, but got &#39;inconsistent,upgrade&#39; " class="issue-link" data-issue-key="LU-5510"><del>LU-5510</del></a> scrub: ldiskfs_create_inode returns locked inode<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 3c357081f5c0af79d76e2f556c14ca74ca47cf3b</p> <p>Landed for 2.7</p> <p>I saw that a few people other than myself hit the problem:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>LustreError: 19224:0:(osd_handler.c:2334:__osd_object_create()) ASSERTION( obj-&gt;oo_inode-&gt;i_state &amp; 8 ) failed </pre> </div></div> <p>that was added as part of this patch. Just adding a comment here to make it clear that you need to do "<tt>make -C ldiskfs clean</tt>" to rebuild the ldiskfs module with the new patch. It seems there is something wrong with the build dependencies in ldiskfs that prevent it from automatically detecting that the source patches have changed since they were applied.</p> <p>I was thinking of disabling this LASSERT() and just handling the case of a returned locked inode, but there isn't really any reason to be compatible with different versions of ldiskfs, since it is always included as part of the Lustre modules (AFAIK only LLNL made their own ldiskfs modules with their kernel and have since moved to ZFS anyway).</p> Development Rank 1|hzwu3j: Rank (Obsolete) 15367 Severity