https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-2547 Lustre <p>== recovery-small test 24a: fsync error (should return error) ======================================== 23:15:59 (1356678959)<br/> multiop /mnt/nbp0-1/d0.recovery-small/d24/f.recovery-small.24a vOwy_wyc<br/> TMPPIPE=/tmp/multiop_open_wait_pipe.9556<br/> fail_loc=0x0<br/> Connected clients:<br/> service331<br/> service331<br/> service332<br/> recovery-small test_24a: @@@@@@ IGNORE (bz5494): multiop didn't fail fsync: rc 0</p> <p>test_logs tarball is attached: recovery-small.24a.tgz</p> Server: 2.1.3-1nasS, centos 6.3, 2.6.32_279.2.1.el6<br/> Client: 2.3.0-1nasC, sles11sp2, 3.0.42_0.7.3<br/> 1 mds, 2 oss, 2 clients.<br/> Test was run from service331.<br/> LU-2547

test: recovery-small test_24a, test_24b: multiop didn't fail fsync: rc 0

Bug Minor Resolved Fixed Niu Yawei Jay Lan revzfs Fri, 28 Dec 2012 19:09:40 +0000 Fri, 22 Dec 2017 10:32:18 +0000 Fri, 2 Sep 2016 04:51:19 +0000 Lustre 2.3.0 Lustre 2.4.0 Lustre 2.9.0 0 9 <p>Lai</p> <p>Could you please look into this one?</p> <p>Thanks</p> <p>Peter</p> <p>Jay, I tested b2_3 branch, it always suceeds, could you point me to the exact version that fails? or a place where I can download?</p> <p>BTW, does this test always fail in your test environment?</p> <p>I tested it three times and all failed in my test environment.</p> <p>The source is at <a href="https://github.com/jlan/lustre-nas" class="external-link" target="_blank" rel="nofollow noopener">https://github.com/jlan/lustre-nas</a>,<br/> branch nas-2.3.0. Tag is 2.3.0-1nasC.</p> <p>Jay, it looks like the code can't compile against 3.0 kernel yet, could you list the patches you've applied?</p> <p>Attached are nas-config and nas-make scripts. You need to modify the nas-config script to specify </p> <ul class="alternate" type="square"> <li>kversion_base to the correct kernel version in your environment,</li> <li>flavor, most likely "default" in your case,</li> <li>make sure kversion and kdir are correct in your environment,</li> <li>"--with-o2ib" parameter to your ofed kernel build environment.</li> <li>This test is not about MPI tests, so comment out module load of mpi tests</li> </ul> <p><a href="https://maloo.whamcloud.com/test_sets/f17e7f9c-6c8c-11e2-91d6-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/f17e7f9c-6c8c-11e2-91d6-52540035b04c</a><br/> pre-2.4 zfs</p> <p>Nathaniel, is the above link for me to read? I can not access that link.</p> <p>Jay, Sorry, that's a link to a failing autotest run.</p> <p>This hasn't failed with ldiskfs in 4wks, but is failing over 50% of the time with zfs.</p> <p>EXCEPT this test for zfs<br/> <a href="http://review.whamcloud.com/6119" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/6119</a></p> <p>It looked like a patch was landed but I saw a zfs fail today that looked exactly like this. </p> <p><a href="https://maloo.whamcloud.com/test_sets/f425e1a6-bc12-11e2-b013-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/f425e1a6-bc12-11e2-b013-52540035b04c</a></p> <p>Patch to EXCEPT 24b also for ZFS<br/> <a href="http://review.whamcloud.com/6587" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/6587</a></p> <p>Got an occurrence with recovery-small/test_24b during <a href="https://maloo.whamcloud.com/test_sets/f43c1ffc-e4ad-11e2-a950-52540035b04c" class="external-link" target="_blank" rel="nofollow noopener">https://maloo.whamcloud.com/test_sets/f43c1ffc-e4ad-11e2-a950-52540035b04c</a>.<br/> When will <a href="http://review.whamcloud.com/6587" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/6587</a> land ?</p> <p>Landed for 2.5</p> <p>Patch for b2_4 <a href="http://review.whamcloud.com/7424" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/7424</a></p> <p>recovery-small test_24a and test_24b are being skipped, the problem was not actually fixed.</p> <p>Please close this ticket since the test was marked "always_except".</p> <p>ok Jay</p> <p>The whole point of the <tt>always_except</tt> label is that it means this test is being skipped, but the original bug has not actually been fixed. This ticket shouldn't be closed until the original problem is fixed (lack of error return to userspace on fsync) and the test is removed from the <tt>ALWAYS_EXCEPT</tt> list in recovery-small.sh. I verified that this test is still being skipped, but only for ZFS MDT.</p> <p>Niu</p> <p>Can you please check to see what needs to happen to get this test re-enabled?</p> <p>Thanks</p> <p>Peter</p> <p>Lustre fsync was semantically wrong before the fix of <a href="https://jira.whamcloud.com/browse/LU-4388" title="fsync on client does not cause OST_SYNCs to be issued" class="issue-link" data-issue-key="LU-4388"><del>LU-4388</del></a>, it didn't sync data on fsync at all (it get the semantic of fsync and fdatasync wrong), so it's not surprise this test fails often.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">00000080:00000001:4.0:1356678959.907382:0:21907:0:(obd_class.h:2061:md_sync()) <span class="code-object">Process</span> leaving (rc=0 : 0 : 0) 00000100:00000001:4.0:1356678959.907382:0:21907:0:(client.c:2323:__ptlrpc_req_finished()) <span class="code-object">Process</span> entered 00000100:00000040:4.0:1356678959.907383:0:21907:0:(client.c:2335:__ptlrpc_req_finished()) @@@ refcount now 0 req@ffff8805bdf93800 x1422578972363063/t0(0) o44-&gt;lustre-MDT0000-mdc-ffff8806f504e800@10.151.25.187@o2ib:12/10 lens 448/408 e 0 to 0 dl 1356678997 ref 1 fl Complete:R/0/0 rc 0/0 00000100:00000001:4.0:1356678959.907386:0:21907:0:(client.c:2245:__ptlrpc_free_req()) <span class="code-object">Process</span> entered 02000000:00000001:4.0:1356678959.907388:0:21907:0:(sec.c:1697:sptlrpc_cli_free_repbuf()) <span class="code-object">Process</span> entered 02000000:00000010:4.0:1356678959.907388:0:21907:0:(sec_null.c:231:null_free_repbuf()) kfreed <span class="code-quote">'req-&gt;rq_repbuf'</span>: 1024 at ffff8805d7a85000. 02000000:00000001:4.0:1356678959.907389:0:21907:0:(sec.c:1711:sptlrpc_cli_free_repbuf()) <span class="code-object">Process</span> leaving 00000020:00000001:4.0:1356678959.907390:0:21907:0:(genops.c:963:class_import_put()) <span class="code-object">Process</span> entered 00000020:00000040:4.0:1356678959.907390:0:21907:0:(genops.c:970:class_import_put()) <span class="code-keyword">import</span> ffff880379630800 refcount=10 obd=lustre-MDT0000-mdc-ffff8806f504e800 00000020:00000001:4.0:1356678959.907391:0:21907:0:(genops.c:979:class_import_put()) <span class="code-object">Process</span> leaving 02000000:00000010:4.0:1356678959.907392:0:21907:0:(sec_null.c:201:null_free_reqbuf()) kfreed <span class="code-quote">'req-&gt;rq_reqbuf'</span>: 512 at ffff8805d8599600. 02000000:00000001:4.0:1356678959.907394:0:21907:0:(sec.c:437:sptlrpc_req_put_ctx()) <span class="code-object">Process</span> entered 02000000:00000001:4.0:1356678959.907394:0:21907:0:(sec.c:453:sptlrpc_req_put_ctx()) <span class="code-object">Process</span> leaving 00000100:00000010:4.0:1356678959.907395:0:21907:0:(client.c:2299:__ptlrpc_free_req()) kfreed <span class="code-quote">'request'</span>: 928 at ffff8805bdf93800. 00000100:00000001:4.0:1356678959.907396:0:21907:0:(client.c:2300:__ptlrpc_free_req()) <span class="code-object">Process</span> leaving 00000100:00000001:4.0:1356678959.907396:0:21907:0:(client.c:2339:__ptlrpc_req_finished()) <span class="code-object">Process</span> leaving (rc=1 : 1 : 1) 00020000:00000002:4.0:1356678959.907397:0:21907:0:(lov_object.c:787:lov_lsm_addref()) lsm ffff8805b90ed640 addref 2 by ffff880382188280. 00020000:00000002:4.0:1356678959.907398:0:21907:0:(lov_object.c:799:lov_lsm_decref()) lsm ffff8805b90ed640 decref 2 by ffff880382188280. 00000080:00000001:4.0:1356678959.907399:0:21907:0:(file.c:2121:ll_fsync()) <span class="code-object">Process</span> leaving (rc=0 : 0 : 0) </pre> </div></div> <p>Above log shows fsync synced meta data only.</p> <p>Given fsync has been fixed since 2.6 (by <a href="https://jira.whamcloud.com/browse/LU-4388" title="fsync on client does not cause OST_SYNCs to be issued" class="issue-link" data-issue-key="LU-4388"><del>LU-4388</del></a>), I think we can simply re-enable these two tests to see if it will go well.</p> <p>Niu Yawei (yawei.niu@intel.com) uploaded a new patch: <a href="http://review.whamcloud.com/22020" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/22020</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-2547" title="test: recovery-small test_24a, test_24b: multiop didn&#39;t fail fsync: rc 0" class="issue-link" data-issue-key="LU-2547"><del>LU-2547</del></a> test: re-enable 24a/b of recovery-small<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: d8c6cdf073a7ae7bd0d9aafb4af288726f4d3bfe</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="http://review.whamcloud.com/22020/" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/22020/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-2547" title="test: recovery-small test_24a, test_24b: multiop didn&#39;t fail fsync: rc 0" class="issue-link" data-issue-key="LU-2547"><del>LU-2547</del></a> test: re-enable 24a/b of recovery-small<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 6912c6f8b1bce483915b3cb386d4695969d29407</p> <p>Landed for 2.9</p> Development End date Fri, 1 Apr 2016 19:09:40 +0000 Rank 1|hzvedr: Rank (Obsolete) 5971 Severity Start date Fri, 28 Dec 2012 19:09:40 +0000