https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-10175 Lustre <p>The Data-on-MDT locking needs lock convert for IBITS lock to drop conflicting bits instead of lock cancel. The main functionality was done in the context of DoM and only for lock used for Data-on-MDT files.</p> <p>Meanwhile it can't be used for all other IBITS lock because of conflicts with ELC. The ELC requires changes to work with lock convert.</p> LU-10175

DoM:Full support for the LDLM lock convert

Technical task LU-10176 Minor Resolved Fixed Mikhail Pershin Mikhail Pershin DoM2 Sun, 4 Sep 2016 15:57:48 +0000 Sat, 25 Aug 2018 07:30:00 +0000 Wed, 4 Jul 2018 13:05:43 +0000 Lustre 2.12.0 0 7 <p>Is it enough to send the lock bits/extent in conflict with the AST to the client when the lock is contended and send the cancelled bits/extent to the server with the cancel? That way, the client can decide whether to cancel the whole lock (e.g. if idle for a long time), or just the conflicting bits (e.g. if actively in use). When the client sends an LDLM_CANCEL to the server, if it cancels all the bits/extent then the server drops the whole lock, otherwise it just drops the contending bits/extent, and leaves the same lock on the client.</p> <p>When a conflicting extent lock gets an AST, the client would need to decide which "end" of the lock should be cancelled so that it keeps a lock with a single contiguous extent, and not two locks with a hole.</p> <p>yes, that is how lock convert works, but ELC cancels locks locally on client and then sends cancels to the server. So if we have lock with several bits plus UPDATE bit and ELC wants to cancel UPDATE bit, then such lock will be cancelled despite other bits. Without ELC, this lock would remain on client until conflict happens on server, then it will get blocking AST and lock_convert which remove only UPDATE bit and keep others. So ELC makes lock_convert useless in many scenarios, especially if there is UPDATE bit set among several others.</p> <p>I think ELC may use lock convert instead of local cancel and this will allow us to use lock convert for all ibits, not just for DOM.</p> <p>Mike Pershin (mike.pershin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/25262" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/25262</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LDEV-459</del></a> ldlm: selective IBITS lock trying<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 17e7d8556f71d7850cb0940ed98fa8a90ab450b3</p> <p>I've just pushed a patch for selective lock try, it is needed as part of DOM work but useful also alone for better IBITS combining. I'd add it prior lock convert code for less conflicts and for better lock convert utilization.</p> <p>Mike Pershin (mike.pershin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/25322" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/25322</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LDEV-459</del></a> ldlm: IBITS lock convert instead of cancel<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 4625cd0b5dbd4f41280dc7fa8ac6c407d540bd91</p> <p>commit IBITS lock convert patch from Data-on-MDT series to the master branch for testing purposes. It has no ELC workaround yet, just adapted to be used without DoM but with previous patch for selective lock trying.</p> <p>Mike Pershin (mike.pershin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/30202" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30202</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LU-10175</del></a> ldlm: IBITS lock convert instead of cancel<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 522b25a09e57bec0453660898e374ca33e496fd5</p> <p>Mike Pershin (mike.pershin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/30491" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30491</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LU-10175</del></a> ldlm: remove obsoleted lock convert code<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 19c844d546cbb227356f50bdb2444f1cfbd8fbaa</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/30491/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30491/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LU-10175</del></a> ldlm: remove obsoleted lock convert code<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: ebba68f378f72107fa51a8002369d1acef7dbedd</p> <p>Full lock convert caused HSM deadlock issues and it seems common HSM problem described in <a href="https://jira.whamcloud.com/browse/LU-5216" title="HSM: restore doesn&#39;t restart for new copytool" class="issue-link" data-issue-key="LU-5216">LU-5216</a>. </p> <p>Full lock convert is stuck on <a href="https://jira.whamcloud.com/browse/LU-5216" title="HSM: restore doesn&#39;t restart for new copytool" class="issue-link" data-issue-key="LU-5216">LU-5216</a> as I can see. Tests 33-36 in sanity-hsm.sh are failed and test 201 experiences deadlock. Lock convert doesn't change anything related to HSM but LDLM and it looks similar to what <a href="https://jira.whamcloud.com/browse/LU-5216" title="HSM: restore doesn&#39;t restart for new copytool" class="issue-link" data-issue-key="LU-5216">LU-5216</a> says about HSM locking problems.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/30202/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/30202/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LU-10175</del></a> ldlm: IBITS lock convert instead of cancel<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 37932c4beb9885593abe63c9e2dc1936648a0b49</p> <p>Landed for 2.12</p> <p>Mike Pershin (mike.pershin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32314" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32314</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LU-10175</del></a> ldlm: handle lock converts in cancel handler<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 00baa72387638913cef71c3037f7894bf0811c6f</p> <p>Reopening as there are more patches to land under this umbrella</p> <p>Mike Pershin (mike.pershin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/32593" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32593</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LU-10175</del></a> ptlrpc: add LOCK_CONVERT connection flag<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: efae969761c5dc27bd481f3f91119fd26a86a31f</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/32593/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32593/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LU-10175</del></a> ptlrpc: add LOCK_CONVERT connection flag<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 44a2092f08ca5f349659680b2c19d55d2365c842</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/32314/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/32314/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-10175" title="DoM:Full support for the LDLM lock convert" class="issue-link" data-issue-key="LU-10175"><del>LU-10175</del></a> ldlm: handle lock converts in cancel handler<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 541902a3f934d0e68368d9698cef38d44c473527</p> <p>Patches have landed for 2.12 for DoM locking.  Additional follow-on work will be created under a separate ticket.</p> Blocker LU-5216 Related LU-9293 LU-3285 LU-11284 LU-11002 LU-9184 Development Rank 1|hzynd3: Rank (Obsolete) 9223372036854775807