https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10670 Lustre <p><a href="https://testing.hpdd.intel.com/test_sets/713fb70e-119d-11e8-a6ad-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/713fb70e-119d-11e8-a6ad-52540065bddc</a></p> <p>It fails very often:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>Error: 'Timeout occurred after 227 mins, last suite running was sanity-flr, restarting cluster to continue tests' Failure Rate: 41.18% of most recent 17 runs, 22 skipped (all branches) </pre> </div></div> <p>On a client:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[10077.749514] Lustre: DEBUG MARKER: == sanity-flr test 43: mirror pick on write ========================================================== 12:14:55 (1518610495) [10320.098013] INFO: task dd:23892 blocked for more than 120 seconds. [10320.114074] "echo 0 &gt; /proc/sys/kernel/hung_task_timeout_secs" disables this message. [10320.116709] dd D ffff88007b96dee0 0 23892 23675 0x00000080 [10320.119330] Call Trace: [10320.125475] [&lt;ffffffff810c6632&gt;] ? default_wake_function+0x12/0x20 [10320.150782] [&lt;ffffffff810bc2d8&gt;] ? __wake_up_common+0x58/0x90 [10320.154162] [&lt;ffffffff816ab8a9&gt;] schedule+0x29/0x70 [10320.170306] [&lt;ffffffff816a92b9&gt;] schedule_timeout+0x239/0x2c0 [10320.176336] [&lt;ffffffffc09f5e88&gt;] ? ptlrpc_set_add_new_req+0xd8/0x150 [ptlrpc] [10320.178829] [&lt;ffffffffc0bd50c0&gt;] ? osc_io_ladvise_end+0x50/0x50 [osc] [10320.181237] [&lt;ffffffffc0a25ffb&gt;] ? ptlrpcd_add_req+0x22b/0x300 [ptlrpc] [10320.183701] [&lt;ffffffffc09fbe99&gt;] ? ptlrpc_request_bufs_pack+0x1d9/0x480 [ptlrpc] [10320.186106] [&lt;ffffffff816abc5d&gt;] wait_for_completion+0xfd/0x140 [10320.188437] [&lt;ffffffff810c6620&gt;] ? wake_up_state+0x20/0x20 [10320.190651] [&lt;ffffffffc0bd5284&gt;] osc_io_setattr_end+0xc4/0x180 [osc] [10320.192955] [&lt;ffffffffc0bd63d0&gt;] ? osc_io_setattr_start+0x260/0x700 [osc] [10320.195231] [&lt;ffffffffc0c28490&gt;] ? lov_io_iter_fini_wrapper+0x50/0x50 [lov] [10320.197659] [&lt;ffffffffc0832e8d&gt;] cl_io_end+0x5d/0x150 [obdclass] [10320.199802] [&lt;ffffffffc0c2856b&gt;] lov_io_end_wrapper+0xdb/0xe0 [lov] [10320.202033] [&lt;ffffffffc0c28bc5&gt;] lov_io_call.isra.5+0x85/0x140 [lov] [10320.204170] [&lt;ffffffffc0c28cb6&gt;] lov_io_end+0x36/0xb0 [lov] [10320.206291] [&lt;ffffffffc0832e8d&gt;] cl_io_end+0x5d/0x150 [obdclass] [10320.208353] [&lt;ffffffffc083551f&gt;] cl_io_loop+0x13f/0xc70 [obdclass] [10320.210509] [&lt;ffffffffc0cd1460&gt;] cl_setattr_ost+0x250/0x3c0 [lustre] [10320.212550] [&lt;ffffffffc0cab495&gt;] ll_setattr_raw+0x1165/0x1270 [lustre] [10320.214631] [&lt;ffffffffc0cab60c&gt;] ll_setattr+0x6c/0xd0 [lustre] [10320.217542] [&lt;ffffffff81220fc1&gt;] notify_change+0x2c1/0x420 [10320.228621] [&lt;ffffffff812b45b6&gt;] ? security_inode_need_killpriv+0x16/0x20 [10320.230605] [&lt;ffffffff81200ad5&gt;] do_truncate+0x75/0xc0 [10320.232485] [&lt;ffffffff81211d97&gt;] do_last+0x627/0x12c0 [10320.234244] [&lt;ffffffff81212af2&gt;] path_openat+0xc2/0x490 [10320.236065] [&lt;ffffffff811af746&gt;] ? do_read_fault.isra.44+0xe6/0x130 [10320.237871] [&lt;ffffffff8121508b&gt;] do_filp_open+0x4b/0xb0 [10320.239642] [&lt;ffffffff8122233a&gt;] ? __alloc_fd+0x8a/0x130 [10320.241313] [&lt;ffffffff81201bc3&gt;] do_sys_open+0xf3/0x1f0 [10320.243068] [&lt;ffffffff816b8945&gt;] ? system_call_after_swapgs+0x172/0x214 [10320.244820] [&lt;ffffffff81201cde&gt;] SyS_open+0x1e/0x20 [10320.246469] [&lt;ffffffff816b89fd&gt;] system_call_fastpath+0x16/0x1b [10320.248096] [&lt;ffffffff816b889d&gt;] ? system_call_after_swapgs+0xca/0x214 </pre> </div></div> LU-10670

sanity-flr test 43 timeout

Bug Blocker Resolved Fixed Zhenyu Xu Mikhail Pershin Thu, 15 Feb 2018 06:41:36 +0000 Tue, 17 Jul 2018 20:34:27 +0000 Tue, 27 Feb 2018 04:24:21 +0000 Lustre 2.11.0 Lustre 2.11.0 0 9 <p>This is a regression failure introduced by patch <a href="https://review.whamcloud.com/30711" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30711</a> for <a href="https://jira.whamcloud.com/browse/LU-10448" title="policy to pick a primary for mirrored write" class="issue-link" data-issue-key="LU-10448"><del>LU-10448</del></a>. The failure occurred 8 times in one day, which is now affecting patch review testing on master branch.</p> <p>Bobi Jam (bobijam@hotmail.com) uploaded a new patch: <a href="https://review.whamcloud.com/31315" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31315</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10670" title="sanity-flr test 43 timeout" class="issue-link" data-issue-key="LU-10670"><del>LU-10670</del></a> test: make sanity-flr test_43 more reliable<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: cc39d8716708fdec6656aa46bb9a36fe8e770c98</p> <p>another on master:<br/> <a href="https://testing.hpdd.intel.com/test_sets/4ff42be0-16cd-11e8-bd00-52540065bddc" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/4ff42be0-16cd-11e8-bd00-52540065bddc</a></p> <p>Another on master:<br/> <a href="https://testing.hpdd.intel.com/test_sessions/c96f08e7-6ef5-4ac4-bc20-dd5f9ef8010a" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/c96f08e7-6ef5-4ac4-bc20-dd5f9ef8010a</a></p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/31315/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/31315/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10670" title="sanity-flr test 43 timeout" class="issue-link" data-issue-key="LU-10670"><del>LU-10670</del></a> test: make sanity-flr test_43 more reliable<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 9af57d0bdce9949dc3fe91817263758b57efbe9b</p> <p>Landed for 2.11</p> <p>Looks like a hit with this fix in:<br/> <a href="https://testing.hpdd.intel.com/test_sessions/0ecb0a78-c3ff-423c-8fdd-0255a7b3a203" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/0ecb0a78-c3ff-423c-8fdd-0255a7b3a203</a></p> <p>the build's parent is e528677e1630093362394ae36d725c321d0da4f2, which does not have this fix.</p> <p>Ah, OK, I will ask him to rebase.</p> Related LU-10250 LU-10448 Development Rank 1|hzzsv3: Rank (Obsolete) 9223372036854775807 Severity