https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-1400 Lustre <p>Looking through obdfilter-survey 1c results I noticed that usually it takes more than 3000s to complete but often it is about 1700s only. All such runs ended up with error in fact but were kept as green so failure is not noticed:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">== obdfilter-survey test 1c: <span class="code-object">Object</span> Storage Targets survey, big batch ================================ 22:23:42 (1336800222) + NETTYPE=tcp thrlo=128 nobjhi=1 thrhi=128 size=8192 <span class="code-keyword">case</span>=disk rslt_loc=/tmp targets=<span class="code-quote">"10.10.4.77:lustre-OST0000 10.10.4.77:lustre-OST0001 10.10.4.77:lustre-OST0002 10.10.4.77:lustre-OST0003 10.10.4.77:lustre-OST0004 10.10.4.77:lustre-OST0005 10.10.4.77:lustre-OST0006"</span> /usr/bin/obdfilter-survey Fri May 11 22:23:48 PDT 2012 Obdfilter-survey <span class="code-keyword">for</span> <span class="code-keyword">case</span>=disk from fat-intel-1vm2 ost 7 sz 58720256K rsz 1024K obj 7 thr 896 write 248.99 ERROR rewrite 119900.58 ERROR read 176.78 [ 0.00, 239.23] done! </pre> </div></div> <p>The same about test 2a which lasts only 2s:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">== obdfilter-survey test 2a: Stripe F/S over the Network == 23:11:25 (1336025485) + NETTYPE=tcp thrlo=8 nobjhi=1 thrhi=16 size=1024 <span class="code-keyword">case</span>=netdisk rslt_loc=/tmp targets=<span class="code-quote">"172.29.3.12:lustre-OST0000 172.29.3.12:lustre-OST0001 172.29.3.12:lustre-OST0002 172.29.3.12:lustre-OST0003 172.29.3.12:lustre-OST0004 172.29.3.12:lustre-OST0005 172.29.3.12:lustre-OST0006"</span> /usr/bin/obdfilter-survey Wed May 2 23:11:25 PDT 2012 Obdfilter-survey <span class="code-keyword">for</span> <span class="code-keyword">case</span>=netdisk from iu-3vm1.lab.whamcloud.com ost 7 sz 7340032K rsz 1024K obj 7 thr 56 write 260954.00 ERROR rewrite 266418.29 ERROR read 269207.02 ERROR ost 7 sz 7340032K rsz 1024K obj 7 thr 112 write 162454.32 ERROR rewrite 162440.28 ERROR read 162512.28 ERROR done! </pre> </div></div> <p><a href="https://maloo.whamcloud.com/sub_tests/ccc4725c-9c20-11e1-8837-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/sub_tests/ccc4725c-9c20-11e1-8837-52540035b04c</a></p> <p>Only interop runs with 2.1 client work normally (check 2a results as 1c test is not in 2.1 yet):<br/> <a href="https://maloo.whamcloud.com/sub_tests/97ddb736-9321-11e1-9e8b-525400d2bfa6" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/sub_tests/97ddb736-9321-11e1-9e8b-525400d2bfa6</a></p> <p>Therefore we have two issues there: 1) obdfilter-survey reporting issue, it stays green always 2) echo client issue causing errors</p> LU-1400

obdfilter-survey test_1c failed but still green in report

Bug Major Resolved Fixed Keith Mannthey Mikhail Pershin Sun, 13 May 2012 04:13:06 +0000 Wed, 17 Apr 2013 12:56:39 +0000 Wed, 29 Aug 2012 13:26:48 +0000 Lustre 2.4.0 0 1 <p>Keith</p> <p>Could you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>Sorry for the delay. <br/> I have submitted a possible fix for the always green issue: <a href="http://review.whamcloud.com/#change,3482" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3482</a>. It needs to be tested but the return codes were not properly handled. </p> <p>I don't quite understand the 2nd error "2) echo client issue causing errors" Can you clarify what you mean? </p> <p>It looks as thought the test itself is working correctly but the /usr/bin/ part is not handling an ENOSPACE error correctly. The systems are running out of desk space but the the /usr/bin/obdfilter-survey script does not pass the failure along. I have added a flag to make the survey itself stop on an error and I am working on testing the change. </p> <p><a href="http://review.whamcloud.com/#change,3591" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3591</a> is looking good for acceptance. </p> <p>There is a patch to fix the all is green issue in master. Please reopen if there is still an error. </p> Related LU-352 Development Rank 1|hzv5tz: Rank (Obsolete) 4479 Severity