https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-17156 Lustre <p>This issue was created by maloo for S Buisson &lt;sbuisson@ddn.com&gt;</p> <p>This issue relates to the following test suite run: <a href="https://testing.whamcloud.com/test_sets/aa93e973-8c6b-4568-a9ce-a2747abd4f7b" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_sets/aa93e973-8c6b-4568-a9ce-a2747abd4f7b</a></p> <p>test_16j failed with the following error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Timeout occurred after 418 minutes, last suite running was sanityn </pre> </div></div> <p>Test session details:<br/> clients: <a href="https://build.whamcloud.com/job/lustre-reviews/99043" class="external-link" target="_blank" rel="nofollow noopener">https://build.whamcloud.com/job/lustre-reviews/99043</a> - 4.18.0-477.21.1.el8_8.x86_64<br/> servers: <a href="https://build.whamcloud.com/job/lustre-reviews/99043" class="external-link" target="_blank" rel="nofollow noopener">https://build.whamcloud.com/job/lustre-reviews/99043</a> - 4.18.0-477.21.1.el8_lustre.x86_64</p> <p>VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV<br/> sanityn test_16j - Timeout occurred after 418 minutes, last suite running was sanityn</p> LU-17156

sanityn test_16j: timeout

Bug Minor Resolved Fixed Arshad Hussain Maloo Fri, 29 Sep 2023 06:29:25 +0000 Sat, 18 Nov 2023 22:03:34 +0000 Sat, 18 Nov 2023 22:03:34 +0000 Lustre 2.16.0 0 5 <p>From: <a href="https://testing.whamcloud.com/test_logs/6315cba4-8e7e-490d-ba12-d4fad74d3784/show_text" class="external-link" target="_blank" rel="nofollow noopener">https://testing.whamcloud.com/test_logs/6315cba4-8e7e-490d-ba12-d4fad74d3784/show_text</a></p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>== sanityn test 16j: race dio with buffered i/o ========== 14:13:36 (1697552016) 1+0 records in 1+0 records out 26214400 bytes (26 MB, 25 MiB) copied, 0.294639 s, 89.0 MB/s CMD: trevis-31vm3 /usr/sbin/lctl get_param -n osd-*.lustre-OST0000 lustre-OST0000_UUID.nonrotational trevis-31vm3: error: get_param: param_path 'lustre-OST0000_UUID/nonrotational': No such file or directory pdsh@trevis-31vm1: trevis-31vm3: ssh exited with exit code 2 /usr/lib64/lustre/tests/test-framework.sh: line 11492: [: : integer expression expected bs: 1024, file_size 26214400 ...</pre> </div></div> <p>It looks like call to zfs_or_rotational() -&gt; ostname_from_index() has returned "lustre-OST0000 lustre-OST0000_UUID" instead of just "lustre-OST0000"</p> <p>"Arshad Hussain &lt;arshad.hussain@aeoncomputing.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/52973" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/52973</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17156" title="sanityn test_16j: timeout" class="issue-link" data-issue-key="LU-17156"><del>LU-17156</del></a> tests: Improve zfs_or_rotational()<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: e4bc1ad986f58e1d29956e274f76a76dc093e0c5</p> <p>It looks like there is a typo in the expansion:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>lctl get_param -n osd-*.lustre-OST0000 lustre-OST0000_UUID.nonrotational </pre> </div></div> <p>There is a space in there, and "<tt>lustre-OST0000</tt>" is shown twice?</p> <p>Yes definitely. This looks like test case issue. I was initially thinking some unusual "lfs osts" output leading to this. Looked at the parsing, all seems correct. So, I put a patch to dump more log on error. (I just saw your review on that, will take care of it). </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>There is a space in there, and "lustre-OST0000" is shown twice?</pre> </div></div> <p>Yes. and also, We should not have "_UUID" string there. It should have been remove.</p> <p>The _UUID string is just a result of the "echo" command only removing the first instance of <tt>_UUID</tt> in the output. When there is only a single OST name printed this will not be a problem.</p> <p>I've updated the patch with the fix. It looks like this has been an issue with the <tt>zfs_or_rotational()</tt> function since it was landed back on 2023-08-24, and it meant that that tests intended <b>not</b> to run on ZFS (because DIO on ZFS is very slow) were being run anyway and sometimes passing, and sometimes timing out.</p> <p>To be clear, if these tests run the full version on ZFS (or maybe even just on a rotational disk), they <b>will</b> time out.  It just takes too long.  So an error in zfs_or_rotational is enough to explain timeouts here.</p> <p> </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>&lt;snip&gt; I've updated the patch with the fix... &lt;snip&gt;</pre> </div></div> <p>Andreas, yes saw that. Thanks again!</p> <p> </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>... It just takes too long.  So an error in zfs_or_rotational is enough to explain timeouts here.</pre> </div></div> <p>Correct. Understood.</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/52973/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/52973/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17156" title="sanityn test_16j: timeout" class="issue-link" data-issue-key="LU-17156"><del>LU-17156</del></a> tests: Improve zfs_or_rotational()<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b0262a00714f2d77b6d6ba745169c8fa18d38b32</p> <p>Landed for 2.16</p> Related LU-13805 Development Rank 1|i03x4f: Rank (Obsolete) 9223372036854775807 Severity