https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13408 Lustre <p>An assertion is triggered, and it mean the request-&gt;rq_transno is 0.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[25892.430672] LustreError: 137-5: fs0a92-OST0006_UUID: not available for connect from 172.16.0.32@o2ib (no target). If you are running an HA pair check that the target is mounted on the other server. [25892.435324] LustreError: Skipped 74 previous similar messages [27887.087931] LustreError: 27219:0:(tgt_main.c:354:tgt_cancel_slc_locks()) ASSERTION( lock-&gt;l_client_cookie != 0 ) failed: [27887.090610] LustreError: 27219:0:(tgt_main.c:354:tgt_cancel_slc_locks()) LBUG </pre> </div></div> LU-13408

tgt_cancel_slc_locks()) ASSERTION( lock->l_client_cookie != 0 )

Bug Critical Resolved Fixed Lai Siyao Lai Siyao Thu, 2 Apr 2020 09:53:24 +0000 Tue, 16 Jun 2020 03:30:42 +0000 Tue, 16 Jun 2020 03:30:42 +0000 Lustre 2.14.0 Lustre 2.14.0 0 6 <p>This same failure made a noticeable appearance in current master-next, but ther's nothing I am able to attribute it to yet.</p> <p>It always happened in sanity 300a</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[30778.381687] Lustre: DEBUG MARKER: == sanity test 300a: basic striped dir sanity test =================================================== 13:43:36 (1585763016) [30779.451744] LustreError: 23319:0:(tgt_main.c:357:tgt_cancel_slc_locks()) ASSERTION( lock-&gt;l_client_cookie != 0 ) failed: [30779.474964] LustreError: 23319:0:(tgt_main.c:357:tgt_cancel_slc_locks()) LBUG [30779.477837] Pid: 23319, comm: jbd2/dm-0-8 3.10.0-7.7-debug #1 SMP Wed Oct 30 08:47:36 EDT 2019 [30779.482233] Call Trace: [30779.483993] [&lt;ffffffffa03b3ddc&gt;] libcfs_call_trace+0x8c/0xc0 [libcfs] [30779.486127] [&lt;ffffffffa03b3e8c&gt;] lbug_with_loc+0x4c/0xa0 [libcfs] [30779.488094] [&lt;ffffffffa07eb08f&gt;] tgt_cancel_slc_locks+0x1cf/0x1e0 [ptlrpc] [30779.491227] [&lt;ffffffffa07ecdb6&gt;] tgt_cb_last_committed+0x116/0x390 [ptlrpc] [30779.496651] [&lt;ffffffffa0ce44db&gt;] osd_trans_commit_cb+0xcb/0x2c0 [osd_ldiskfs] [30779.500512] [&lt;ffffffffa0c86fa4&gt;] ldiskfs_journal_commit_callback+0x84/0xc0 [ldiskfs] [30779.504687] [&lt;ffffffffa0b14e9b&gt;] jbd2_journal_commit_transaction+0x186b/0x1ca0 [jbd2] [30779.514340] [&lt;ffffffffa0b1a87d&gt;] kjournald2+0xcd/0x280 [jbd2] [30779.516415] [&lt;ffffffff810b8254&gt;] kthread+0xe4/0xf0 [30779.518263] [&lt;ffffffff817e0ddd&gt;] ret_from_fork_nospec_begin+0x7/0x21 [30779.522814] [&lt;ffffffffffffffff&gt;] 0xffffffffffffffff [30779.528997] Kernel panic - not syncing: LBUG </pre> </div></div> <p>I got like 5 crashes (have crashdumps) on the first run then it subdued and on testing restart got another crash and then it went silent again for now.</p> <p>Alex Zhuravlev (bzzz@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/38177" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38177</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13408" title="tgt_cancel_slc_locks()) ASSERTION( lock-&gt;l_client_cookie != 0 )" class="issue-link" data-issue-key="LU-13408"><del>LU-13408</del></a> tests: pass localrecov to MGS<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: c89337791dc9f026dfbbf2ac695dac7254a9800e</p> <p>the patch above is a workaround. the root cause is that MDT got mounted w/o localrecov option and then the client's request had no transno,<br/> which in turn used to track commit status of cross-mdt operations.<br/> I <em>guess</em> this can be solved by a per-tx callback, instead of using transno.</p> <p>Alex, wouldn't it be better to handle this by ignoring the "localrecov" behavior for MDT and MGT mounts, rather than changing the transaction callback?</p> <p>I <em>think</em> this issue is fixed by patch <a href="https://review.whamcloud.com/38138" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38138</a> "<tt><a href="https://jira.whamcloud.com/browse/LU-13402" title="sanity test_252: Invalid number of mdtlov clients returned by /usr/sbin/lr_reader" class="issue-link" data-issue-key="LU-13402"><del>LU-13402</del></a> target: never exclude MDT/OST from last_rcvd</tt>" but I'd like Alex to confirm before this issue is closed.</p> <p>Andreas, no, these are different problems. <em>this</em> issue happens because <em>client</em> (running on MDS) is excluded from recovery, thus doesn't generate a transno which is used to track committness (in turn used to cancel locks).</p> <p>+1 on master running sanity test_103a in my VM.<br/> +1 on master running sanity test_103b in my VM.</p> <p>Seems like this is easy for me to reproduce if there is something that you think will fix this properly.</p> <p>LDLM lock is handled in MDT layer, while transaction is in MDD layer, it's layer violation to mix them together. IMO if "req_transno" of an operation is not 0, it's not needed to enforce Commit-on-Sharing for such operation.</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/38855" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38855</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13408" title="tgt_cancel_slc_locks()) ASSERTION( lock-&gt;l_client_cookie != 0 )" class="issue-link" data-issue-key="LU-13408"><del>LU-13408</del></a> mdt: don't save remote lock if req_transno is 0<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: af33439d1e383d10b226870fd533f33ffac7f078</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38855/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38855/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13408" title="tgt_cancel_slc_locks()) ASSERTION( lock-&gt;l_client_cookie != 0 )" class="issue-link" data-issue-key="LU-13408"><del>LU-13408</del></a> target: update in-memory per client data<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 300858ccfcd00b52663de45e0bb472012242f342</p> <p>Landed for 2.14</p> Duplicate LU-13402 Related Development Rank 1|i00wun: Rank (Obsolete) 9223372036854775807 Severity