https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10734 Lustre <p>sanity test_160g - User cl8 still found in changelog_users<br/> ^^^^^^^^^^^^^ DO NOT REMOVE LINE ABOVE ^^^^^^^^^^^^^</p> <p>This issue was created by maloo for Bob Glossman &lt;bob.glossman@intel.com&gt;</p> <p>This issue relates to the following test suite run: <br/> <a href="https://testing.hpdd.intel.com/test_sets/5a8495f4-1bfa-11e8-a6ad-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/5a8495f4-1bfa-11e8-a6ad-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/34e243bc-1be3-11e8-a7cd-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/34e243bc-1be3-11e8-a7cd-52540065bddc</a></p> <p>test_160g failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>User cl8 still found in changelog_users </pre> </div></div> <p>This may be a dup of <a href="https://jira.whamcloud.com/browse/LU-9624" title="enable sanity.sh test_160a failures for DNE" class="issue-link" data-issue-key="LU-9624"><del>LU-9624</del></a><br/> I can't tell if it is so I am raising a fresh ticket.<br/> Will let somebody else decide if it's a dup or not.</p> LU-10734

sanity test_160g: User cl8 still found in changelog_users

Bug Minor Resolved Fixed Bruno Faccini Maloo sles12 suse Tue, 27 Feb 2018 20:15:10 +0000 Thu, 19 Jul 2018 22:24:27 +0000 Wed, 18 Jul 2018 12:34:08 +0000 Lustre 2.11.0 Lustre 2.12.0 0 9 <p>more on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/9ac85f8a-1cac-11e8-a7cd-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/9ac85f8a-1cac-11e8-a7cd-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/e7203b64-1cac-11e8-a7cd-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/e7203b64-1cac-11e8-a7cd-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/dc78ca60-1cba-11e8-a6ad-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/dc78ca60-1cba-11e8-a6ad-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/27be5f90-1cc9-11e8-bd00-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/27be5f90-1cc9-11e8-bd00-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/1645ec0c-1ccd-11e8-a7cd-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/1645ec0c-1ccd-11e8-a7cd-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/fc6956e8-1d62-11e8-a10a-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/fc6956e8-1d62-11e8-a10a-52540065bddc</a></p> <p>These fails only seen on master. Probably because sanity, 160g only exists on master.</p> <p>Well, it is strange that it seems to only fail running with SLES, and looks like it started to fail after my 1st patch for <a href="https://jira.whamcloud.com/browse/LU-10680" title="MDT becoming unresponsive in 2.10.3" class="issue-link" data-issue-key="LU-10680"><del>LU-10680</del></a> (<a href="https://review.whamcloud.com/31347/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31347/</a>) has landed, when it should fix an issue with my previous patch for <a href="https://jira.whamcloud.com/browse/LU-7340" title="ChangeLogs catalog full condition should be handled more gracefully" class="issue-link" data-issue-key="LU-7340"><del>LU-7340</del></a> which has introduced sanity/test_160<span class="error">&#91;f,g&#93;</span> tests !!... I think I need to reproduce and debug it this way.</p> <p>more on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/f9311b56-209a-11e8-a4b1-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/f9311b56-209a-11e8-a4b1-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/c68848ae-20b4-11e8-9ec4-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/c68848ae-20b4-11e8-9ec4-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/50a68744-20d3-11e8-a6ca-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/50a68744-20d3-11e8-a6ca-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/9d2125f0-2157-11e8-9ec4-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/9d2125f0-2157-11e8-9ec4-52540065bddc</a><br/> <a href="https://testing.hpdd.intel.com/test_sets/bbef9e12-2184-11e8-9ec4-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/bbef9e12-2184-11e8-9ec4-52540065bddc</a></p> <p>after many similar fails here is a SLES test run that did NOT hit the failure:<br/> <a href="https://testing.hpdd.intel.com/test_sessions/ba2847b6-e445-448d-882d-356fca02b96e" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/ba2847b6-e445-448d-882d-356fca02b96e</a></p> <p>Don't know what the diff is between runs that fail and those that don't.<br/> Noting this instance that didn't fail in the hope that it may be of some use.</p> <p>here is a similar fail seen on el7, not on SLES at all.<br/> proof this problem isn't SLES only.</p> <p><a href="https://testing.hpdd.intel.com/test_sets/b3cb95da-20dd-11e8-a4b1-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/b3cb95da-20dd-11e8-a4b1-52540065bddc</a></p> <p>+1 on master, all with DNE<br/> testing.hpdd.intel.com/test_sessions/7a5adbc7-2d4b-425a-9e71-a4674823a0df<br/> testing.hpdd.intel.com/test_sessions/a9ae8e29-d45d-49b6-a639-a6fba84f5dfc</p> <p>Having a better look to the recent changes that may have introduced this regression, I think that "a37134d <a href="https://jira.whamcloud.com/browse/LU-9624" title="enable sanity.sh test_160a failures for DNE" class="issue-link" data-issue-key="LU-9624"><del>LU-9624</del></a> tests: fix pre-DNE test exceptions/llog usage" could better be the cause of it.</p> <p>Hope to get more about this soon now.</p> <p>It looks like this failure relates to the landing of patch <a href="https://review.whamcloud.com/27535" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27535</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-9624" title="enable sanity.sh test_160a failures for DNE" class="issue-link" data-issue-key="LU-9624"><del>LU-9624</del></a> tests: fix pre-DNE test exceptions/llog usage</tt>". It didn't fail during normal testing, but I guess SLES is not part of regular testing.</p> <p>Note also that with patch <a href="https://review.whamcloud.com/31552" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31552</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-10680" title="MDT becoming unresponsive in 2.10.3" class="issue-link" data-issue-key="LU-10680"><del>LU-10680</del></a> mdd: disable changelog garbage collection by default</tt>" <tt>test_160f</tt> and <tt>test_160g</tt> need to be modified to set <tt>changelog_gc=1</tt> at the start of each test, and remove the tests from <tt>ALWAYS_EXCEPT</tt> so that the tests will run properly.</p> <p>&gt; It didn't fail during normal testing, but I guess SLES is not part of regular testing.</p> <p>Well, it is tested regularly, but due to the round robin system used for pre-landing review test runs, it is not guaranteed to run before everything lands unless people proactively request this with test parameters.</p> <p>Eh eh, after taking some time to think about it, I was wondering if the only regression/side effect coming from patch <a href="https://review.whamcloud.com/27535" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27535</a> ("a37134d <a href="https://jira.whamcloud.com/browse/LU-9624" title="enable sanity.sh test_160a failures for DNE" class="issue-link" data-issue-key="LU-9624"><del>LU-9624</del></a> tests: fix pre-DNE test exceptions/llog usage"), that we strongly suspect to be the cause of these failures, is to have slightly reduced the execution/elapsed time of sanity/test_160g script's beginning/prologue that may now take less than the 2 seconds of delay interval between 2x garbage-collection thread runs (as it has just run in sanity/test_160f, when sanity.sh is being fully executed during auto-tests) being used/configured ("changelog_min_gc_interval=2").<br/> And this seems to be confirmed during my reproducer testing.</p> <p>So a simple "sleep 2" at the beginning of sanity/test_160g should fix this problem.</p> <p>Faccini Bruno (bruno.faccini@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/31604" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31604</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10734" title="sanity test_160g: User cl8 still found in changelog_users" class="issue-link" data-issue-key="LU-10734"><del>LU-10734</del></a> tests: ensure current GC interval is over<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 72032b016ea8ab62cc681e72b5565ba207a6c316</p> <p>This fail is blocked for now. test 160g was added to ALWAYS_EXCEPT in a patch landed to master for <a href="https://jira.whamcloud.com/browse/LU-10680" title="MDT becoming unresponsive in 2.10.3" class="issue-link" data-issue-key="LU-10680"><del>LU-10680</del></a>. May need to look for similar fails if and when test 160g is taken back out of ALWAYS_EXCEPT.</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/31604/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31604/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10734" title="sanity test_160g: User cl8 still found in changelog_users" class="issue-link" data-issue-key="LU-10734"><del>LU-10734</del></a> tests: ensure current GC interval is over<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 821087e65882a9885964ed07d6f2a630dfb599d5</p> <p>Landed for 2.12</p> Related LU-10680 LU-11161 LU-9624 Development Rank 1|hzztg7: Rank (Obsolete) 9223372036854775807 Severity