https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10321 Lustre <p>MDT 2 (soak-10) fails over to soak-11, with errors</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">Dec 2 07:27:00 soak-11 kernel: LustreError: 2976:0:(llog_osd.c:960:llog_osd_next_block()) soaked-MDT0003-osp-MDT0002: missed desired record? 2 &gt; 1 Dec 2 07:27:00 soak-11 kernel: LustreError: 2976:0:(lod_dev.c:419:lod_sub_recovery_thread()) soaked-MDT0003-osp-MDT0002 getting update log failed: rc = -2 Dec 2 07:27:00 soak-11 kernel: LustreError: 2976:0:(lod_dev.c:419:lod_sub_recovery_thread()) Skipped 3 previous similar messages Dec 2 07:27:01 soak-11 kernel: LustreError: 2381:0:(mdt_open.c:1167:mdt_cross_open()) soaked-MDT0002: [0x280002b4c:0xa44:0x0] doesn't exist!: rc = -14 Dec 2 07:27:02 soak-11 kernel: Lustre: 2977:0:(ldlm_lib.c:2059:target_recovery_overseer()) recovery is aborted, evict exports in recovery Dec 2 07:27:02 soak-11 kernel: Lustre: 2977:0:(ldlm_lib.c:2059:target_recovery_overseer()) Skipped 2 previous similar messages Dec 2 07:27:02 soak-11 kernel: Lustre: soaked-MDT0002: disconnecting 31 stale clients </pre> </div></div> <p>Soak attempts a umount which hangs:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">2017-12-02 07:27:16,430:fsmgmt.fsmgmt:INFO Unmounting soaked-MDT0002 on soak-11 ... soak-11 Dec 2 07:30:16 soak-11 kernel: INFO: task umount:3039 blocked <span class="code-keyword">for</span> more than 120 seconds. Dec 2 07:30:16 soak-11 kernel: <span class="code-quote">"echo 0 &gt; /proc/sys/kernel/hung_task_timeout_secs"</span> disables <span class="code-keyword">this</span> message. Dec 2 07:30:16 soak-11 kernel: umount D ffff8803c81f4008 0 3039 3037 0x00000080 Dec 2 07:30:16 soak-11 kernel: ffff8803ce3afa30 0000000000000086 ffff88081fa50000 ffff8803ce3affd8 Dec 2 07:30:16 soak-11 kernel: ffff8803ce3affd8 ffff8803ce3affd8 ffff88081fa50000 ffff8803c81f4000 Dec 2 07:30:16 soak-11 kernel: ffff8803c81f4004 ffff88081fa50000 00000000ffffffff ffff8803c81f4008 Dec 2 07:30:16 soak-11 kernel: Call Trace: Dec 2 07:30:16 soak-11 kernel: [&lt;ffffffff816aa489&gt;] schedule_preempt_disabled+0x29/0x70 Dec 2 07:30:16 soak-11 kernel: [&lt;ffffffff816a83b7&gt;] __mutex_lock_slowpath+0xc7/0x1d0 Dec 2 07:30:16 soak-11 kernel: [&lt;ffffffff816a77cf&gt;] mutex_lock+0x1f/0x2f Dec 2 07:30:16 soak-11 kernel: [&lt;ffffffffc14560c7&gt;] lfsck_stop+0x167/0x4e0 [lfsck] Dec 2 07:30:16 soak-11 kernel: [&lt;ffffffff810c4832&gt;] ? default_wake_function+0x12/0x20 Dec 2 07:30:16 soak-11 kernel: [&lt;ffffffff811e0593&gt;] ? __kmalloc+0x1e3/0x230 Dec 2 07:30:16 soak-11 kernel: [&lt;ffffffffc1625aa6&gt;] mdd_iocontrol+0x96/0x16a0 [mdd] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffffc0ec9619&gt;] ? lprocfs_counter_add+0xf9/0x160 [obdclass] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffffc1500fc1&gt;] mdt_device_fini+0x71/0x920 [mdt] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffffc0ed6911&gt;] class_cleanup+0x971/0xcd0 [obdclass] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffffc0ed8cad&gt;] class_process_config+0x19cd/0x23b0 [obdclass] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffffc0dc6bc7&gt;] ? libcfs_debug_msg+0x57/0x80 [libcfs] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffffc0ed9856&gt;] class_manual_cleanup+0x1c6/0x710 [obdclass] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffffc0f07fee&gt;] server_put_super+0x8de/0xcd0 [obdclass] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffff81203692&gt;] generic_shutdown_super+0x72/0x100 Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffff81203a62&gt;] kill_anon_super+0x12/0x20 Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffffc0edc152&gt;] lustre_kill_super+0x32/0x50 [obdclass] Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffff81203e19&gt;] deactivate_locked_super+0x49/0x60 Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffff81204586&gt;] deactivate_super+0x46/0x60 Dec 2 07:30:17 soak-11 kernel: [&lt;ffffffff812217cf&gt;] cleanup_mnt+0x3f/0x80 Dec 2 07:30:18 soak-11 kernel: [&lt;ffffffff81221862&gt;] __cleanup_mnt+0x12/0x20 Dec 2 07:30:18 soak-11 kernel: [&lt;ffffffff810ad275&gt;] task_work_run+0xc5/0xf0 Dec 2 07:30:18 soak-11 kernel: [&lt;ffffffff8102ab62&gt;] do_notify_resume+0x92/0xb0 Dec 2 07:30:18 soak-11 kernel: [&lt;ffffffff816b533d&gt;] int_signal+0x12/0x17 Dec 2 07:30:19 soak-11 kernel: LustreError: 11-0: soaked-OST0016-osc-MDT0002: operation ost_connect to node 192.168.1.106@o2ib failed: rc = -114 </pre> </div></div> <p>This wedges soak, no further faults are attempted, jobs stop scheduling. <br/> This happened over the weekend. Dumped Lustre logs, forced a crash dump. <br/> Logs, crash info attached.<br/> Full crash dump is available on Spirit. </p> Soak stress cluster MLNX networking stack. LU-10321

MDS - umount hangs during failback

Bug Critical Resolved Fixed nasf Cliff White soak Mon, 4 Dec 2017 17:54:10 +0000 Fri, 2 Feb 2018 18:56:50 +0000 Thu, 4 Jan 2018 16:36:21 +0000 Lustre 2.10.2 Lustre 2.11.0 Lustre 2.10.4 0 7 <p>Maybe a lfsck start/recovery dealdock.</p> <p>umount is waiting in <tt>lfsck_stop()</tt> on <tt>li_mutex</tt>. <tt>tgt_recover_2</tt> is in <tt>lfsck_start()</tt> holding <tt>li_mutex</tt>. Possibly waiting on <tt>lfsck_layout</tt> (pid 3013) which is sending notifications but is stuck waiting for import recovery.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>00100000:10000000:17.0:1512199622.506187:0:3013:0:(lfsck_engine.c:1576:lfsck_assistant_engine()) soaked-MDT0002-osd: lfsck_layout LFSCK assistant thread start 00000100:00080000:17.0:1512199622.506589:0:3013:0:(client.c:1563:ptlrpc_send_new_req()) @@@ req from PID 0 waiting for recovery: (FULL != CONNECTING) req@ffff8803ee66b900 x1585651536798656/t0(0) o1101-&gt;soaked-OST0016-osc-MDT0002@192.168.1.106@o2ib:28/4 lens 320/224 e 0 to 0 dl 0 ref 1 fl Rpc:W/0/ffffffff rc 0/-1 </pre> </div></div> <p>Relevant backtraces:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>lfsck_namespace S ffff8803c81f4000 0 3012 2 0x00000080^M [&lt;----------------&gt;] schedule+0x29/0x70^M [&lt;----------------&gt;] lfsck_assistant_engine+0x118d/0x20c0 [lfsck]^M [&lt;----------------&gt;] ? dequeue_task_fair+0x41e/0x660^M [&lt;----------------&gt;] ? __schedule+0x39d/0x8b0^M [&lt;----------------&gt;] ? wake_up_state+0x20/0x20^M [&lt;----------------&gt;] ? lfsck_master_engine+0x12f0/0x12f0 [lfsck]^M [&lt;----------------&gt;] kthread+0xcf/0xe0^M [&lt;----------------&gt;] ? insert_kthread_work+0x40/0x40^M [&lt;----------------&gt;] ret_from_fork+0x58/0x90^M [&lt;----------------&gt;] ? insert_kthread_work+0x40/0x40^M lfsck S ffff8803f6dc9800 0 3010 2 0x00000080^M [&lt;----------------&gt;] schedule+0x29/0x70^M [&lt;----------------&gt;] lfsck_start_assistant+0x1d5/0x320 [lfsck]^M [&lt;----------------&gt;] ? dt_read+0x14/0x50 [obdclass]^M [&lt;----------------&gt;] ? lfsck_layout_load_bitmap+0x4bb/0x710 [lfsck]^M [&lt;----------------&gt;] ? wake_up_state+0x20/0x20^M [&lt;----------------&gt;] lfsck_layout_master_prep+0xfe/0x250 [lfsck]^M [&lt;----------------&gt;] lfsck_master_engine+0x176/0x12f0 [lfsck]^M [&lt;----------------&gt;] ? __switch_to+0xd7/0x510^M [&lt;----------------&gt;] ? __schedule+0x39d/0x8b0^M [&lt;----------------&gt;] ? lfsck_master_oit_engine+0x1150/0x1150 [lfsck]^M [&lt;----------------&gt;] kthread+0xcf/0xe0^M [&lt;----------------&gt;] ? insert_kthread_work+0x40/0x40^M [&lt;----------------&gt;] ret_from_fork+0x58/0x90^M [&lt;----------------&gt;] ? insert_kthread_work+0x40/0x40^M lfsck_layout S 000000000000f908 0 3013 2 0x00000080^M [&lt;----------------&gt;] schedule+0x29/0x70^M [&lt;----------------&gt;] schedule_timeout+0x174/0x2c0^M [&lt;----------------&gt;] ? internal_add_timer+0x70/0x70^M [&lt;----------------&gt;] ? ptlrpc_init_rq_pool+0x110/0x110 [ptlrpc]^M [&lt;----------------&gt;] ptlrpc_set_wait+0x4c0/0x910 [ptlrpc]^M [&lt;----------------&gt;] ? wake_up_state+0x20/0x20^M [&lt;----------------&gt;] lfsck_assistant_notify_others+0x110f/0x1390 [lfsck]^M [&lt;----------------&gt;] ? libcfs_debug_msg+0x57/0x80 [libcfs]^M [&lt;----------------&gt;] ? lfsck_master_engine+0x12f0/0x12f0 [lfsck]^M [&lt;----------------&gt;] lfsck_assistant_engine+0x140/0x20c0 [lfsck]^M [&lt;----------------&gt;] ? check_preempt_curr+0x78/0xa0^M [&lt;----------------&gt;] ? dequeue_task_fair+0x41e/0x660^M [&lt;----------------&gt;] ? __switch_to+0xd7/0x510^M [&lt;----------------&gt;] ? __schedule+0x39d/0x8b0^M [&lt;----------------&gt;] ? lfsck_master_engine+0x12f0/0x12f0 [lfsck]^M [&lt;----------------&gt;] kthread+0xcf/0xe0^M [&lt;----------------&gt;] ? insert_kthread_work+0x40/0x40^M [&lt;----------------&gt;] ret_from_fork+0x58/0x90^M [&lt;----------------&gt;] ? insert_kthread_work+0x40/0x40^M tgt_recover_2 S ffff8808067d9fa0 0 2977 2 0x00000080^M [&lt;----------------&gt;] schedule+0x29/0x70^M [&lt;----------------&gt;] lfsck_start+0xf25/0x17c0 [lfsck]^M [&lt;----------------&gt;] ? llog_key_init+0x20/0xd0 [obdclass]^M [&lt;----------------&gt;] ? wake_up_state+0x20/0x20^M [&lt;----------------&gt;] mdd_iocontrol+0x3d3/0x16a0 [mdd]^M [&lt;----------------&gt;] ? osp_key_init+0x3b/0xd0 [osp]^M [&lt;----------------&gt;] mdt_postrecov+0x148/0x1d0 [mdt]^M [&lt;----------------&gt;] mdt_obd_postrecov+0x8d/0xb0 [mdt]^M [&lt;----------------&gt;] target_recovery_thread+0xaf2/0x1360 [ptlrpc]^M [&lt;----------------&gt;] ? __schedule+0x39d/0x8b0^M [&lt;----------------&gt;] ? replay_request_or_update.isra.21+0x8c0/0x8c0 [ptlrpc]^M [&lt;----------------&gt;] kthread+0xcf/0xe0^M [&lt;----------------&gt;] ? insert_kthread_work+0x40/0x40^M [&lt;----------------&gt;] ret_from_fork+0x58/0x90^M [&lt;----------------&gt;] ? insert_kthread_work+0x40/0x40^M umount D ffff8803c81f4008 0 3039 3037 0x00000080^M [&lt;----------------&gt;] schedule_preempt_disabled+0x29/0x70^M [&lt;----------------&gt;] __mutex_lock_slowpath+0xc7/0x1d0^M [&lt;----------------&gt;] mutex_lock+0x1f/0x2f^M [&lt;----------------&gt;] lfsck_stop+0x167/0x4e0 [lfsck]^M [&lt;----------------&gt;] ? default_wake_function+0x12/0x20^M [&lt;----------------&gt;] ? __kmalloc+0x1e3/0x230^M [&lt;----------------&gt;] mdd_iocontrol+0x96/0x16a0 [mdd]^M [&lt;----------------&gt;] ? lprocfs_counter_add+0xf9/0x160 [obdclass]^M [&lt;----------------&gt;] mdt_device_fini+0x71/0x920 [mdt]^M [&lt;----------------&gt;] class_cleanup+0x971/0xcd0 [obdclass]^M [&lt;----------------&gt;] class_process_config+0x19cd/0x23b0 [obdclass]^M [&lt;----------------&gt;] ? libcfs_debug_msg+0x57/0x80 [libcfs]^M [&lt;----------------&gt;] class_manual_cleanup+0x1c6/0x710 [obdclass]^M [&lt;----------------&gt;] server_put_super+0x8de/0xcd0 [obdclass]^M [&lt;----------------&gt;] generic_shutdown_super+0x72/0x100^M [&lt;----------------&gt;] kill_anon_super+0x12/0x20^M [&lt;----------------&gt;] lustre_kill_super+0x32/0x50 [obdclass]^M [&lt;----------------&gt;] deactivate_locked_super+0x49/0x60^M [&lt;----------------&gt;] deactivate_super+0x46/0x60^M [&lt;----------------&gt;] cleanup_mnt+0x3f/0x80^M [&lt;----------------&gt;] __cleanup_mnt+0x12/0x20^M [&lt;----------------&gt;] task_work_run+0xc5/0xf0^M [&lt;----------------&gt;] do_notify_resume+0x92/0xb0^M [&lt;----------------&gt;] int_signal+0x12/0x17^M </pre> </div></div> <p>Fan Yong, does this look familiar?</p> <p>According to current implementation (master), the LFSCK notify RPC is interruptible, so would you please to try "kill -9 3013" to check whether the layout LFSCK assistant engine can be interrupted? If yes, then the lfsck_start can go ahead (exit with failure), and then the lfsck_stop will not be blocked.</p> <p>Okay, so you want lfsck_layout process killed if this happens again?</p> <p>Yes, please kill the blocked (on recovery) LFSCK assistant thread, such as lfsck_layout or lfsck_namespace. It is expected to work, otherwise, we need to find out the solution.</p> <p>It does not work.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">[root@soak-8 ~]# ps -ef |grep lfsck root 6695 6693 0 04:35 ? 00:00:00 bash -c ??lctl lfsck_start -M soaked-MDT0000 -s 1000 -t all -A ??<span class="code-keyword">if</span> [[ $? != 0 ]]; then ???lctl lfsck_start -M soaked-MDT0000 -s 1000 -t namespace,layout -A ???<span class="code-keyword">if</span> [[ $? != 0 ]]; then ????lctl lfsck_start -M soaked-MDT0000 -s 1000 -t namespace -A ???fi ??fi root 6713 6695 0 04:35 ? 00:00:00 lctl lfsck_start -M soaked-MDT0000 -s 1000 -t all -A root 6714 2 0 04:35 ? 00:00:00 [lfsck] root 6716 2 0 04:35 ? 00:00:00 [lfsck_layout] root 53709 53676 0 16:09 pts/0 00:00:00 lctl lfsck_stop -M soaked-MDT0000 root 53748 53717 0 16:10 pts/1 00:00:00 grep --color=auto lfsck [root@soak-8 ~]# kill -9 6716 [root@soak-8 ~]# kill -9 6716 [root@soak-8 ~]# ps -ef |grep lfsck root 6695 6693 0 04:35 ? 00:00:00 bash -c ??lctl lfsck_start -M soaked-MDT0000 -s 1000 -t all -A ??<span class="code-keyword">if</span> [[ $? != 0 ]]; then ???lctl lfsck_start -M soaked-MDT0000 -s 1000 -t namespace,layout -A ???<span class="code-keyword">if</span> [[ $? != 0 ]]; then ????lctl lfsck_start -M soaked-MDT0000 -s 1000 -t namespace -A ???fi ??fi root 6713 6695 0 04:35 ? 00:00:00 lctl lfsck_start -M soaked-MDT0000 -s 1000 -t all -A root 6714 2 0 04:35 ? 00:00:00 [lfsck] root 6716 2 0 04:35 ? 00:00:00 [lfsck_layout] root 53709 53676 0 16:09 pts/0 00:00:00 lctl lfsck_stop -M soaked-MDT0000 root 53754 53717 0 16:11 pts/1 00:00:00 grep --color=auto lfsck </pre> </div></div> <p>Process is un-killable, soak-8 is wedged. I can leave it like this if you want to login.</p> <p>Just login soak-8, but seems reboot.<br/> This situation is more complex than <a href="https://jira.whamcloud.com/browse/LU-6684" title="lctl lfsck_stop hangs" class="issue-link" data-issue-key="LU-6684"><del>LU-6684</del></a>. According to current (master) implementation, when lfsck_stop, it will send SIGINT signal to the LFSCK engines, then even if the LFSCK engines are blocked by some RPCs, it still can be waken up and exit. But in this case, the lfsck_stop was blocked on the mutex, then it has no chance to send the SIGINT single.</p> <p>Sorry, I'll let you know when the next hang happens. I should have dumped stacks before the reboot.</p> <p>I think i have re-produced the hang, I'll leave it like this until you have a look. <br/> Sigh, it finally exited. Trying again.</p> <p>Fan Yong (fan.yong@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/30420" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30420</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10321" title="MDS - umount hangs during failback " class="issue-link" data-issue-key="LU-10321"><del>LU-10321</del></a> lfsck: allow to stop the in-starting lfsck<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 7817928a62c8a212f5c2b040587042c193bd5c1a</p> <p>Fan Yong (fan.yong@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/30422" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30422</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10321" title="MDS - umount hangs during failback " class="issue-link" data-issue-key="LU-10321"><del>LU-10321</del></a> lfsck: allow to stop the in-starting lfsck<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: 162e2f5c04b32904053ac31795450eb9da521c00</p> <p>I think there is already an option <tt>no scrub</tt> to prevent Oi scrub from starting. </p> <p>However, it would also be useful if it was possible to force LFSCK to abort the running scan at startup, in case there is a problem like this at a customer. Is “<tt>no scrub</tt>” along with “<tt>lctl lfsck_stop —reset</tt>” enough to kill the running LFSCK, and erase the saved state?</p> <p>For this case, it is NOT the LFSCK auto-resumed from the last check-point, instead, it was started manually (or by scripts) after MDT failover. So no related with mount option.</p> <p>Hit the problem again last night. System completed a failback, MDTs are all mounted in the proper place, lfsck is wedged. Will leave the system if you wish to have a look. </p> <p>Dumped lustre logs from all MDS, output in /scratch/results/soak/soak-X-hung.lfsck.Dec08.txt<br/> Dumped stacks on all MDS, output in console logs. Crash-dumped all MDS, results on Spirit. <br/> Restarting. </p> <p>Had a simular hang on soak-3 and soak-6 (OSS) during umount. Dumped lustre-logs, attached. Also dumped stack traces and crash dumped both nodes, output available on spirit. </p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=cliffw" class="user-hover" rel="cliffw">cliffw</a>, as you said on Skype, there are multiple issues during current Spirit tests. The original hung happened on the MDT because of the blocked uninterrupted LFSCK. The patch 30420 is used for resolving such trouble. But it does not means the Spirit will not hung after applying the patch, because some other issues may also block the system. For the new hung, it happened on the OST side, different from the original LFSCK hung. So would you please to check whether the original LFSCK hung issue resolved or not? Thanks!</p> <p>We have not seen lfsck hangs with MDT failover after the patch. The two hangs occurred with OST failover. After the most recent hang, i was able to reboot/remount the system and then I could start and stop lfsck without the hang. </p> <p>According to the logs, three were new LFSCK started just after lfsck_stop during the MDT umount. Then nobody will stop the new triggered LFSCK as to the MDT cannot umount. I will make patch to resolve the race condition.</p> <p>Fan Yong (fan.yong@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/30513" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30513</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10321" title="MDS - umount hangs during failback " class="issue-link" data-issue-key="LU-10321"><del>LU-10321</del></a> lfsck: not start lfsck during umount<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 6263064b369dd4fddbb0dfa9ab49013a0d791629</p> <p>Fan Yong (fan.yong@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/30514" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30514</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10321" title="MDS - umount hangs during failback " class="issue-link" data-issue-key="LU-10321"><del>LU-10321</del></a> lfsck: not start lfsck during umount<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: 2022d417ddaf663dc7addb5389acade0390996e5</p> <p>We just hit this again on master, so we need a version of the patch for master. </p> <p>Okay, we have an IB build so we will test this.</p> <p>We are still hitting this issue, with the patch. Soak-10 just hit it, crash dump vmcore-dmesg and lustre-log attached</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/30420/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30420/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10321" title="MDS - umount hangs during failback " class="issue-link" data-issue-key="LU-10321"><del>LU-10321</del></a> lfsck: allow to stop the in-starting lfsck<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 9c9a05fee6c0fce557dfa578ff7116b905d4e00a</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/30513/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30513/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10321" title="MDS - umount hangs during failback " class="issue-link" data-issue-key="LU-10321"><del>LU-10321</del></a> lfsck: not start lfsck during umount<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 0cabe06be1fcc6c3ff690889b1e86bbe4690b854</p> <p>Patches have been landed to master, other LFSCK start trouble will be handled via <a href="https://jira.whamcloud.com/browse/LU-10419" title="LFSCK fails to start, hangs systems. " class="issue-link" data-issue-key="LU-10419"><del>LU-10419</del></a></p> <p>John L. Hammond (john.hammond@intel.com) merged in patch <a href="https://review.whamcloud.com/30422/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30422/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10321" title="MDS - umount hangs during failback " class="issue-link" data-issue-key="LU-10321"><del>LU-10321</del></a> lfsck: allow to stop the in-starting lfsck<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: 147b2b4177f9f8d3c4407cd74bb1b97a5c5d7f53</p> <p>John L. Hammond (john.hammond@intel.com) merged in patch <a href="https://review.whamcloud.com/30514/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30514/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10321" title="MDS - umount hangs during failback " class="issue-link" data-issue-key="LU-10321"><del>LU-10321</del></a> lfsck: not start lfsck during umount<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: e82d8ad284fd37bbc4863723aa42c927d456fae6</p> Related LU-6684 Development Rank 1|hzzonr: Rank (Obsolete) 9223372036854775807 Severity