https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-4442 Lustre <p>This issue was created by maloo for sarah &lt;sarah@whamcloud.com&gt;</p> <p>This issue relates to the following test suite run: <a href="http://maloo.whamcloud.com/test_sets/930d9194-74df-11e3-96b0-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">http://maloo.whamcloud.com/test_sets/930d9194-74df-11e3-96b0-52540035b04c</a>.</p> <p>The sub-test test_7g failed with the following error:</p> <blockquote> <p>Test 7g.3 failed</p></blockquote> <p>test log shows:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>CMD: client-31vm7 /usr/sbin/lctl list_param osp.*osc*.old_sync_processed 2&gt; /dev/null osp.lustre-OST0000-osc-MDT0000.old_sync_processed osp.lustre-OST0001-osc-MDT0000.old_sync_processed osp.lustre-OST0002-osc-MDT0000.old_sync_processed osp.lustre-OST0003-osc-MDT0000.old_sync_processed osp.lustre-OST0004-osc-MDT0000.old_sync_processed osp.lustre-OST0005-osc-MDT0000.old_sync_processed osp.lustre-OST0006-osc-MDT0000.old_sync_processed wait mds1 secs maximumly for client-31vm7 mds-ost sync done. /usr/lib64/lustre/tests/test-framework.sh: line 2135: [: mds1: integer expression expected CMD: client-31vm7 /usr/sbin/lctl get_param -n osp.*osc*.old_sync_processed 1 1 1 1 1 1 1 recovery node iozone not done in mds1 sec. replay-vbr test_7g: @@@@@@ FAIL: Test 7g.3 failed </pre> </div></div> client and server: lustre-master build 1823 RHEL6 ldiskfs LU-4442

Failure on test suite replay-vbr test_7g: Test 7g.3 failed

Bug Minor Resolved Fixed Emoly Liu Maloo mn1 mn4 Mon, 6 Jan 2014 17:59:34 +0000 Tue, 14 Jul 2015 02:55:16 +0000 Tue, 11 Feb 2014 07:38:18 +0000 Lustre 2.1.6 Lustre 2.6.0 Lustre 2.5.1 Lustre 2.4.3 Lustre 2.6.0 Lustre 2.5.1 0 9 <p>there's certainly some parsing error somewhee that makes us pick out of mds name instead of some timeout and so thngs go downhill fro there:</p> <p>/usr/lib64/lustre/tests/test-framework.sh: line 2135: [: mds1: integer expression expected</p> <p>Emoly</p> <p>Could you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>I can reproduce it easily. I will investigate and fix it.</p> <p>patch at: <a href="http://review.whamcloud.com/8796" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8796</a>, which fixes the test script issue.</p> <p>Lustre build: <a href="http://build.whamcloud.com/job/lustre-reviews/20841/" class="external-link" target="_blank" rel="nofollow noopener">http://build.whamcloud.com/job/lustre-reviews/20841/</a><br/> Distro/Arch: SLES11SP3/x86_64 (both server and client, kernel version: 3.0.101-0.8)</p> <p>The same failure occurred:<br/> <a href="https://maloo.whamcloud.com/test_sets/43ce86d2-798b-11e3-a27b-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/43ce86d2-798b-11e3-a27b-52540035b04c</a></p> <p>The maloo test report <a href="https://maloo.whamcloud.com/test_logs/2a98cbc0-7bfa-11e3-a7b6-52540035b04c/show_text" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_logs/2a98cbc0-7bfa-11e3-a7b6-52540035b04c/show_text</a> shows that test_7g has another problem besides test script issue to be fixed by <a href="http://review.whamcloud.com/8796" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8796</a>.</p> <p>I will investigate and provide another patch for it.</p> <p>By searching maloo, I notice this error has occurred since Dec. 21, and finally I find it is related to <a href="https://jira.whamcloud.com/browse/LU-3528" title="move directory entries cache from llite to MDC" class="issue-link" data-issue-key="LU-3528"><del>LU-3528</del></a> <a href="http://review.whamcloud.com/8371" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8371</a> .</p> <p>I am working on the patch.</p> <p>The root cause of this failure is that since mdt_object_exists() was added to mdt_reint_link() in <a href="http://review.whamcloud.com/#/c/8371" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#/c/8371</a>, if the child object doesn't exist, there is no chance to do object version check and client1 will not be evicted. </p> <p>I create the following two patches to fix this problem, and I am not sure which is better:</p> <ul class="alternate" type="square"> <li><a href="http://review.whamcloud.com/8972" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8972</a> is to do mdt_version_get_check() before calling mdt_object_exists(), so that the client can be evicted due to (EOVERFLOW)"Version mismatch" as before;</li> <li><a href="http://review.whamcloud.com/8973" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8973</a> is to change test_7g script to check the different return value for different MDS version.</li> </ul> <p>Tappro, could you please give any advice? Thanks.</p> <p>Thanks, Tappro, I saw your choice of <a href="http://review.whamcloud.com/8973" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8973</a> .</p> <p>Both patches have been landed to 2.6.</p> <p>Patches for Lustre b2_5 branch:<br/> <a href="http://review.whamcloud.com/9289" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9289</a><br/> <a href="http://review.whamcloud.com/9290" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9290</a></p> <p>Landing <a href="http://review.whamcloud.com/8371" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/8371</a> on Lustre b2_5 build #25 also caused this regression failure on Lustre b2_5 branch:<br/> <a href="https://maloo.whamcloud.com/test_sets/0103e6d0-977c-11e3-b941-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/0103e6d0-977c-11e3-b941-52540035b04c</a><br/> <a href="https://maloo.whamcloud.com/test_sets/c79df196-97b8-11e3-acb5-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/c79df196-97b8-11e3-acb5-52540035b04c</a></p> <p>Also hit on interop test between 2.5.1 server and master client:</p> <p><a href="https://maloo.whamcloud.com/test_sets/8e4f6b3a-b244-11e3-a93f-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/8e4f6b3a-b244-11e3-a93f-52540035b04c</a></p> <blockquote><p>Also hit on interop test between 2.5.1 server and master client:</p></blockquote> <p>This is because <a href="http://review.whamcloud.com/9213" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9213</a> was landed on Lustre b2_5 branch. We need change Lustre version number 2.5.52 to 2.5.1 in replay-vbr test_7g().<br/> Patch for master branch: <a href="http://review.whamcloud.com/9986" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9986</a></p> <p>Back-ported patch for Lustre b2_4 branch: <a href="http://review.whamcloud.com/9987" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9987</a><br/> Back-ported patch for Lustre b2_1 branch: <a href="http://review.whamcloud.com/9988" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/9988</a></p> Development Rank 1|hzwc8v: Rank (Obsolete) 12186 Severity