https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-8351 Lustre <p>ptlrpc_import_delay_req() refuses to delay blocking asts when import<br/> is not in LUSTRE_IMP_FULL yet. That leads to client eviction assuming<br/> that it failed to respond.</p> <p>Solution: allow delays for blocking asts being resent.</p> LU-8351

ptlrpc_import_delay_req() refuses to delay blocking asts when import is not in LUSTRE_IMP_FULL yet

Bug Major Resolved Fixed Oleg Drokin Artem Blagodarenko patch Wed, 29 Jun 2016 13:05:15 +0000 Sat, 17 Dec 2016 14:12:17 +0000 Sat, 17 Dec 2016 14:12:17 +0000 Lustre 2.10.0 0 5 <p>Artem Blagodarenko (artem.blagodarenko@seagate.com) uploaded a new patch: <a href="http://review.whamcloud.com/21065" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/21065</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-8351" title="ptlrpc_import_delay_req() refuses to delay blocking asts when import is not in LUSTRE_IMP_FULL yet" class="issue-link" data-issue-key="LU-8351"><del>LU-8351</del></a> ptlrpc: allow blocking asts to be delayed<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 44dc760a25d313cd96f9dbd5071ff6018326eae6</p> <blockquote> <p>Was it considered that the alternative is the server is stuck waiting for a lock from unavailable client for a long time?<br/> What's the plan there?<br/> Before the patch: Send AST, once there is timeout, retry a few more times, then evict client.<br/> Now: Try to send AST, it gets blocked... wait...? (for how long) and then what?<br/> What if the client never reappears, are we waiting for it to be evicted by the ping evictor or something?<br/> What about all the other clients that could get blocked on this lock meanwhile?</p></blockquote> <p>I believe that all the above will remain as it is now.</p> <p>The only situation the patch is to address arises when a server handles reconnect from a client and when there is blocking ast for that client to be resend.</p> <p>The server enters target_handle_connect() and eventually rev_import_reconnect(), where the reverse import is set to LUSTRE_IMP_RECOVER, and then ptlrpc_import_recovery_state_machine() is called to do resends.<br/> ptlrpc_resend() sets rq_resend flag for each request on the sending list and wakeup corresponding ptlrpc_check_set-s. Then the import state is set to LUSTRE_IMP_FULL. </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">if</span> (imp-&gt;imp_state == LUSTRE_IMP_RECOVER) { struct ptlrpc_connection *conn = imp-&gt;imp_connection; rc = ptlrpc_resend(imp); <span class="code-keyword">if</span> (rc) GOTO(out, rc); IMPORT_SET_STATE(imp, LUSTRE_IMP_FULL); </pre> </div></div> <p>There is a import’s lock race window between ptlrpc_resend() and IMPORT_SET_STATE(imp, LUSTRE_IMP_FULL).<br/> If awaken ptlrpc_set_check()&#45;&gt;ptlrpc_import_delay_req() locks the import before import’s state gets set to LUSTRE_IMP_FULL, it sees req-&gt;rq_send_state != imp-&gt;imp_state and sets *status to -EWOULDBLOCK(-EAGAIN). </p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> } <span class="code-keyword">else</span> <span class="code-keyword">if</span> (req-&gt;rq_send_state != imp-&gt;imp_state) { <span class="code-comment">/* invalidate in progress - any requests should be drop */</span> <span class="code-keyword">if</span> (atomic_read(&amp;imp-&gt;imp_inval_count) != 0) { DEBUG_REQ(D_ERROR, req, <span class="code-quote">"invalidate in flight"</span>); *status = -EIO; } <span class="code-keyword">else</span> <span class="code-keyword">if</span> (imp-&gt;imp_dlm_fake || req-&gt;rq_no_delay) { *status = -EWOULDBLOCK; </pre> </div></div> <p>After that the request goes to interpret state and ldlm_handle_ast_error()&#45;&gt;ldlm_failed_ast() evicts the client as req-&gt;rq_status is -EAGAIN.</p> <blockquote> <p>Also, the import on the server is not a real import, so how could it be non-full?</p></blockquote> <p>The server is handling reconnect. The state of the reverse import is changed in rev_import_reconnect().</p> <blockquote> <p>this probably needs an interop check too? I.e. the change is on the server, but we always run with client scriptss, so we need to ensure the server has this fix before running this test?</p></blockquote> <p>I do not know. The test is attached just to illustrate the problem. If you would like older servers testing to not fail you might want to just add the test to exception list.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/21065/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/21065/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-8351" title="ptlrpc_import_delay_req() refuses to delay blocking asts when import is not in LUSTRE_IMP_FULL yet" class="issue-link" data-issue-key="LU-8351"><del>LU-8351</del></a> ptlrpc: allow blocking asts to be delayed<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: bde948c9cf11793d8e7fef59ea2a10bb2d684bf6</p> <p>Landed for 2.10</p> Duplicate Development Rank 1|hzyg67: Rank (Obsolete) 9223372036854775807 Severity