https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-8502 Lustre <p>This issue was created by maloo for Kit Westneat &lt;kit.westneat@gmail.com&gt;</p> <p>Test timed out with a lot of stack traces related to mgs_ir_fini_fs:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">[17640.500244] [&lt;ffffffff8163ba29&gt;] schedule+0x29/0x70 [17640.500244] [&lt;ffffffff81639719&gt;] schedule_timeout+0x209/0x2d0 [17640.500244] [&lt;ffffffff811bfad9&gt;] ? discard_slab+0x39/0x50 [17640.500244] [&lt;ffffffff81632d4d&gt;] ? __slab_free+0x253/0x277 [17640.500244] [&lt;ffffffff8163bdf6&gt;] wait_for_completion+0x116/0x170 [17640.500244] [&lt;ffffffff810b88c0&gt;] ? wake_up_state+0x20/0x20 [17640.500244] [&lt;ffffffffa0cb3a0e&gt;] mgs_ir_fini_fs+0x27e/0x2ec [mgs] [17640.500244] [&lt;ffffffffa0ca0361&gt;] mgs_free_fsdb+0x41/0x8e0 [mgs] [17640.500244] [&lt;ffffffffa0ca97d2&gt;] mgs_cleanup_fsdb_list+0x52/0x70 [mgs] [17640.500244] [&lt;ffffffffa0c8fa87&gt;] mgs_device_fini+0x97/0x5b0 [mgs] [17640.500244] [&lt;ffffffffa07d088c&gt;] class_cleanup+0x94c/0xd80 [obdclass] [17640.500244] [&lt;ffffffffa07d3606&gt;] class_process_config+0x2226/0x2f60 [obdclass] [17640.500244] [&lt;ffffffff811c2483&gt;] ? __kmalloc+0x1f3/0x230 [17640.500244] [&lt;ffffffffa07cd6cb&gt;] ? lustre_cfg_new+0x8b/0x400 [obdclass] [17640.500244] [&lt;ffffffffa07d442f&gt;] class_manual_cleanup+0xef/0x810 [obdclass] [17640.500244] [&lt;ffffffffa0802560&gt;] server_put_super+0xb20/0xcd0 [obdclass] [17640.500244] [&lt;ffffffff811e1096&gt;] generic_shutdown_super+0x56/0xe0 [17640.500244] [&lt;ffffffff811e1472&gt;] kill_anon_super+0x12/0x20 [17640.500244] [&lt;ffffffffa07d7c92&gt;] lustre_kill_super+0x32/0x50 [obdclass] [17640.500244] [&lt;ffffffff811e1829&gt;] deactivate_locked_super+0x49/0x60 [17640.500244] [&lt;ffffffff811e1e26&gt;] deactivate_super+0x46/0x60 [17640.500244] [&lt;ffffffff811fed95&gt;] mntput_no_expire+0xc5/0x120 [17640.500244] [&lt;ffffffff811ffecf&gt;] SyS_umount+0x9f/0x3c0 </pre> </div></div> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">[17640.500244] mgs_lustre_noti S ffff88004c88dc00 0 21703 2 0x00000080 [17640.500244] ffff88004c83fbb0 0000000000000046 ffff88004c88dc00 ffff88004c83ffd8 [17640.500244] ffff88004c83ffd8 ffff88004c83ffd8 ffff88004c88dc00 ffff88004f27a800 [17640.500244] ffff88004c88dc00 0000000000000000 ffffffffa09d4b90 ffff88004c88dc00 [17640.500244] Call Trace: [17640.500244] [&lt;ffffffffa09d4b90&gt;] ? ldlm_completion_ast_async+0x300/0x300 [ptlrpc] [17640.500244] [&lt;ffffffff8163ba29&gt;] schedule+0x29/0x70 [17640.500244] [&lt;ffffffffa09d540d&gt;] ldlm_completion_ast+0x62d/0x910 [ptlrpc] [17640.500244] [&lt;ffffffff810b88c0&gt;] ? wake_up_state+0x20/0x20 [17640.500244] [&lt;ffffffffa0c8e8f1&gt;] mgs_completion_ast_generic+0xb1/0x1d0 [mgs] [17640.500244] [&lt;ffffffffa0c8ea23&gt;] mgs_completion_ast_ir+0x13/0x20 [mgs] [17640.500244] [&lt;ffffffffa09d7ab0&gt;] ldlm_cli_enqueue_local+0x230/0x860 [ptlrpc] [17640.500244] [&lt;ffffffffa0c8ea10&gt;] ? mgs_completion_ast_generic+0x1d0/0x1d0 [mgs] [17640.500244] [&lt;ffffffffa09da820&gt;] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [17640.500244] [&lt;ffffffffa0c93ddc&gt;] mgs_revoke_lock+0x1ec/0x370 [mgs] [17640.500244] [&lt;ffffffffa09da820&gt;] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [17640.500244] [&lt;ffffffffa0c8ea10&gt;] ? mgs_completion_ast_generic+0x1d0/0x1d0 [mgs] [17640.500244] [&lt;ffffffffa0cb0462&gt;] mgs_ir_notify+0x142/0x2a0 [mgs] [17640.500244] [&lt;ffffffff810b88c0&gt;] ? wake_up_state+0x20/0x20 [17640.500244] [&lt;ffffffffa0cb0320&gt;] ? lprocfs_ir_set_state+0x170/0x170 [mgs] [17640.500244] [&lt;ffffffff810a5aef&gt;] kthread+0xcf/0xe0 [17640.500244] [&lt;ffffffff810a5a20&gt;] ? kthread_create_on_node+0x140/0x140 [17640.500244] [&lt;ffffffff816469d8&gt;] ret_from_fork+0x58/0x90 [17640.500244] [&lt;ffffffff810a5a20&gt;] ? kthread_create_on_node+0x140/0x140 </pre> </div></div> <p>This issue relates to the following test suite run: <a href="https://testing.hpdd.intel.com/test_sets/89fac3d8-634d-11e6-906c-5254006e85c2" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sets/89fac3d8-634d-11e6-906c-5254006e85c2</a>.</p> LU-8502

replay-vbr: umount hangs waiting for mgs_ir_fini_fs

Bug Major Resolved Cannot Reproduce Jinshan Xiong Maloo Tue, 16 Aug 2016 02:10:58 +0000 Thu, 4 Jan 2018 18:36:29 +0000 Mon, 18 Dec 2017 20:53:16 +0000 Lustre 2.10.0 Lustre 2.11.0 Lustre 2.11.0 0 7 <p>Seeing a lot of test set PASSes followed by test set test_0a TIMEOUTs:</p> <p>replay-dual PASS, replay-dual TIMEOUT<br/> <a href="https://testing.hpdd.intel.com/test_sessions/279f69ac-eda3-4fd2-a1e9-f9135f7c0d66" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/279f69ac-eda3-4fd2-a1e9-f9135f7c0d66</a></p> <p>replay-ost-single PASS, replay-dual TIMEOUT<br/> <a href="https://testing.hpdd.intel.com/test_sessions/4a7fd4bc-e055-44c6-afac-97569c944b02" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/4a7fd4bc-e055-44c6-afac-97569c944b02</a></p> <p>replay-dual PASS, replay-single TIMEOUT<br/> <a href="https://testing.hpdd.intel.com/test_sessions/bce01895-216d-4460-b513-24c7b02ef25e" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/bce01895-216d-4460-b513-24c7b02ef25e</a></p> <p>In this case, replay-dual was the last test set run:</p> <p><a href="https://testing.hpdd.intel.com/test_sessions/2862ae67-6628-41c8-9561-e586502f1a13" class="external-link" target="_blank" rel="nofollow noopener">https://testing.hpdd.intel.com/test_sessions/2862ae67-6628-41c8-9561-e586502f1a13</a></p> <p>It passed all subtests, and then hung on the umount. The test set was then marked TIMEOUT. <span class="nobr"><a href="https://jira.whamcloud.com/secure/attachment/26238/26238_trevis-41vm7.log" title="trevis-41vm7.log attached to LU-8502">trevis-41vm7.log^{<img class="rendericon" src="https://jira.whamcloud.com/images/icons/link_attachment_7.gif" height="7" width="7" align="absmiddle" alt="" border="0"/>}</a></span> </p> <p>Jinshan</p> <p>Could you please advise on this one?</p> <p>Thanks</p> <p>Peter</p> <p>I suspect this is the same issue as <a href="https://jira.whamcloud.com/browse/LU-7372" title="replay-dual test_26: test failed to respond and timed out" class="issue-link" data-issue-key="LU-7372"><del>LU-7372</del></a>, the patch is located at: <a href="https://review.whamcloud.com/17853" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/17853</a></p> <p>James Nunez (james.a.nunez@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/27255" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27255</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-8502" title="replay-vbr: umount hangs waiting for mgs_ir_fini_fs " class="issue-link" data-issue-key="LU-8502"><del>LU-8502</del></a> test: Baseline test failure rates<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: bc450568faa8fb98d9650d9b834a83f8f5e2efb8</p> <p>James Nunez (james.a.nunez@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/27256" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27256</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-8502" title="replay-vbr: umount hangs waiting for mgs_ir_fini_fs " class="issue-link" data-issue-key="LU-8502"><del>LU-8502</del></a> test: Run <a href="https://jira.whamcloud.com/browse/LU-7372" title="replay-dual test_26: test failed to respond and timed out" class="issue-link" data-issue-key="LU-7372"><del>LU-7372</del></a> patch against failing tests<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 0f498ea376a184aadfb502fde9fee0b79319f6ea</p> <p>I've reviewed the last month of replay-vbr test 1b hangs and they are all during interop testing with master (future 2.11) clients and previous versions of Lustre servers including the b2_8 and b2_9 branches. Thus, this issues looks like it is fixed.</p> Related LU-7372 LU-9113 Development Rank 1|hzykov: Rank (Obsolete) 9223372036854775807 Severity