https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-1575 Lustre <p>The goal of this ticket is to add the support for the patched clients 1.8 to be correctly handled on the 2.x fixed side.</p> <p>The patch has to contains next two parts:<br/> 1) client side change (for the <a href="https://bugzilla.lustre.org/attachment.cgi?id=30902" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.lustre.org/attachment.cgi?id=30902</a>) - here we need to add new flag for 1.8 clients which has the change from this patch (OBD_CONNECT_FLOCK_OWNER).<br/> 2) server side change for the 2.x <a href="http://review.whamcloud.com/#patch,unified,2193,2,lustre/ldlm/ldlm_lock.c" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#patch,unified,2193,2,lustre/ldlm/ldlm_lock.c</a> patch and extend it with the flag OBD_CONNECT_FLOCK_OWNER checking and call the ldlm_flock_policy_wire21_to_local() function for this case.</p> LU-1575

Add flock policy support

Improvement Minor Resolved Fixed Oleg Drokin Iurii Golovach client server Wed, 27 Jun 2012 09:30:03 +0000 Wed, 28 Oct 2015 00:29:31 +0000 Wed, 28 Oct 2015 00:29:31 +0000 Lustre 2.2.0 Lustre 2.3.0 Lustre 2.1.1 Lustre 1.8.6 0 8 <p><a href="http://review.whamcloud.com/#change,3202" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3202</a> - patch which introduce new flag on the server side (2.x).</p> <p><a href="http://review.whamcloud.com/3203" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3203</a> - patch which introduce new flag on the client side (1.8).</p> <p>Making this a 2.3 blocker until the interop patches are landed. The original patch in <a href="https://jira.whamcloud.com/browse/LU-1137" title="flock owner incorrectly handled on the server side" class="issue-link" data-issue-key="LU-1137"><del>LU-1137</del></a> was landed in git commit 7526d21040f4064f83fc4350cd58a8f39de913b3 (v2_2_50_0-50-g7526d21) so the interop issue is only in the current master branch.</p> <p>Patches with OBD_CONNECT_FLAG_OWNER were introduced sepatately according the request from Andreas at <a href="https://jira.whamcloud.com/browse/LU-1770" title="Introducing OBD_CONNECT_FLOCK_OWNER flag" class="issue-link" data-issue-key="LU-1770"><del>LU-1770</del></a>.</p> <p>Short explanation of the patch is next:</p> <p>As you may see at 1.8 there is only introdusion OBD_CONNECT_OWNER flag and sending it to the server. Defines where OBD_CONNECT_OWNER is included are placed at the lustre_idl.h and there is no functional changes on the client side at all.</p> <p>There is a change on the server side.</p> <p>Before the patch:</p> <p>new_client = (exp-&gt;exp_connect_flags &amp; OBD_CONNECT_FULL20) != 0;<br/> if (new_client)<br/> convert = ldlm_policy_wire21_to_local<span class="error">&#91;type - LDLM_MIN_TYPE&#93;</span>;<br/> else<br/> convert = ldlm_policy_wire18_to_local<span class="error">&#91;type - LDLM_MIN_TYPE&#93;</span>;</p> <p>After the patch:</p> <p>new_client = (exp-&gt;exp_connect_flags &amp; OBD_CONNECT_FULL20) != 0;<br/> if (new_client)<br/> convert = ldlm_policy_wire21_to_local<span class="error">&#91;type - LDLM_MIN_TYPE&#93;</span>;<br/> else if ((exp-&gt;exp_connect_flags &amp; OBD_CONNECT_FLOCK_OWNER) != 0 &amp;&amp;<br/> type == LDLM_FLOCK)<br/> convert = ldlm_policy_wire21_to_local<span class="error">&#91;type - LDLM_MIN_TYPE&#93;</span>;<br/> else<br/> convert = ldlm_policy_wire18_to_local<span class="error">&#91;type - LDLM_MIN_TYPE&#93;</span>;</p> <p>As you may see the new check was introduced:</p> <p>else if ((exp-&gt;exp_connect_flags &amp; OBD_CONNECT_FLOCK_OWNER) != 0 &amp;&amp;<br/> type == LDLM_FLOCK)<br/> convert = ldlm_policy_wire21_to_local<span class="error">&#91;type - LDLM_MIN_TYPE&#93;</span>;</p> <p>There are few ideas in this lines.</p> <p>1) The funtional change idea: </p> <p>Without this line server recognize the 1.8 client like a client with old flock policy (wich is native for 1.8.x).<br/> But patch <a href="https://bugzilla.lustre.org/attachment.cgi?id=30902" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.lustre.org/attachment.cgi?id=30902</a> introduce the 2.x flock policy in 1.8 clients.</p> <p>The difference between 1.8 and 2.x flock policy handling is in assigning flocks to the OWNER instead of using PID.</p> <p>The issue comes when 1.8 clients (patched with <a href="https://bugzilla.lustre.org/attachment.cgi?id=30902" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.lustre.org/attachment.cgi?id=30902</a>) try to work with 2.x server.</p> <p>Since 2.x server recognize them like 1.8 clients and start working with PID field instead of OWNER field.</p> <p>2.x server uses the next function:</p> <p>void ldlm_flock_policy_wire18_to_local(const ldlm_wire_policy_data_t *wpolicy,<br/> ldlm_policy_data_t *lpolicy)<br/> {<br/> .....<br/> lpolicy-&gt;l_flock.owner = wpolicy-&gt;l_flock.lfw_pid;<br/> }</p> <p>While it should use the next one:<br/> void ldlm_flock_policy_wire21_to_local(const ldlm_wire_policy_data_t *wpolicy,<br/> ldlm_policy_data_t *lpolicy)<br/> {<br/> .....<br/> lpolicy-&gt;l_flock.owner = wpolicy-&gt;l_flock.lfw_owner;<br/> }</p> <p>That's all about functional changes.</p> <p>Now two ideas in code:</p> <p>2) It may looks like OBD_CONNECT_FULL20 flag can be used and there is no needs in a new flag.<br/> This is incorrect, since OBD_CONNECT_FULL20 usage adds some additional 2.x related funcitonality which is currently unsupported in 1.8 code.</p> <p>3) sepatare "else if" is used:<br/> ((exp-&gt;exp_connect_flags &amp; OBD_CONNECT_FLOCK_OWNER) != 0 &amp;&amp;<br/> type == LDLM_FLOCK)</p> <p>The reason is that we have use 2.x functionality for the LDLM_FLOCK only. And use 1.8 functionality for other cases.</p> <p>Andreas, please, let me know if there are any questions which this test doesn't cover.</p> <p>Repeating some of the comments from the inspection of the b1_8 version of the patch, so that everyone is clear on what is going on here:</p> <ul class="alternate" type="square"> <li>if the reservation of OBD_CONNECT_FLOCK_OWNER flag is landing, it should be landed on all of the branches. There are currently patches for b1_8 and master (should be included into b2_3), but patches are also necessary for b2_1 and b2_2 to avoid potential conflicts if someone is developing a new feature against b2_1 and doesn't see this bit reserved.</li> <li>the b1_8 version of the patch <a href="http://review.whamcloud.com/3203" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3203</a> cannot be landed to our b1_8 Git repo as is, since we do not (AFAICS) include the b=22040/att=30902 change yet. Otherwise, any clients built from that branch (e.g. from Git or 1.8.8) would incorrectly advertise their support for FLOCK_OWNER when they don't actually handle this feature correctly.</li> <li>the master version of the fix (<a href="http://review.whamcloud.com/3202" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3202</a>) needs to be rebased on top of the feature reservation patch (<a href="http://review.whamcloud.com/3722" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3722</a>).</li> <li>I see that <a href="https://jira.whamcloud.com/browse/LU-1137" title="flock owner incorrectly handled on the server side" class="issue-link" data-issue-key="LU-1137"><del>LU-1137</del></a> (<a href="http://review.whamcloud.com/2193" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/2193</a>) was included into our 2.1.2 release, but since the code is conditional upon the OBD_CONNECT_FULL20 flag, this will only continue working as long as 1.8.x clients do not have the att=30902 fix applied.</li> </ul> <p>So to move forward, either b1_8 should only get the flag reservation patch (<a href="http://review.whamcloud.com/3723" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3723</a>), or the att=30902 patch needs to be submitted to b1_8 merged together with change 3203 to actually use the flag, so that there is no time when the b1_8 clients are sending the OWNER field without also connecting with the OBD_CONNECT_FLOCK_OWNER feature flag. At the same time (i.e. blocking the 2.1.3 release, which is basically out the door already), b2_1 would also need to be patched to understand this flag, and it would cause 2.1.0/2.1.1/2.1.2/2.2 servers to break against patched 1.8.8 clients.</p> <p>It isn't clear to me whether there is any benefit of submitting the "fix" to b1_8 at all, since the easiest thing to do would be to leave all 1.8 clients unpatched and handle this at the server. Is there a functional benefit for 1.8 clients if they DO set the OWNER field, as opposed to the 2.x servers with <a href="https://jira.whamcloud.com/browse/LU-1137" title="flock owner incorrectly handled on the server side" class="issue-link" data-issue-key="LU-1137"><del>LU-1137</del></a> applied handling this themselves?</p> <p>Without a clear functional benefit to applying the original att=30902 change to b1_8, I don't see any value in going down this road at all for our 1.8 clients.</p> <p>I understand that Xyratex has patched 1.8.x clients in the field using this code, and they will be applying (or already have) <a href="http://review.whamcloud.com/3203" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3203</a> in order to declare their use of the flock OWNER field. I think it also makes sense to apply an updated version of 3202/3722 to master (and b2_3 if already forked) and b2_1, to avoid potential interop issues with patched b1_8 clients (Xyratex/Oracle/Cray/etc). However, I don't yet see any benefit for applying att=30902 to our b1_8 at this time, and definite interoperability problems would be introduced for existing deployments.</p> <p>Having a subtest added to sanity.sh test_105 that actually exercised this code in master would help avoid potential interoperability issues in the future, and would ensure that this code is exercised during our release interoperability testing, which is run regularly by autotest.</p> <p>There is a benefit to b1_8, actually more than one:<br/> 1. flock (the bsd whole-file locks, as opposed to posix locks we usually call by this name) starts to really work (things like working across processes and such.<br/> 2. locking starts to work in case of nfs4 reexport.</p> <p>That said, att. 3203 all by itself is harmful, it also needs the actual logic implemented, plus on top of that logic, it needs a bit of interop to now include owner info if the server did not acknowledge the owner flag.</p> <p>Can <a href="http://review.whamcloud.com/3202" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3202</a> land to master?</p> <p>Xyratex-bug-id: <a href="http://jira-nss.xy01.xyratex.com:8080/browse/MRP-489" class="external-link" target="_blank" rel="nofollow noopener">MRP-489</a></p> <p>Status update.<br/> 1) Issue summary:<br/> During usage <a href="https://bugzilla.lustre.org/attachment.cgi?id=30902" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.lustre.org/attachment.cgi?id=30902</a> with 1.8 customer met with an issue when 1.8 client can't be recognized on 2.x server.<br/> 2) Resolution:<br/> To fix this issue next patches were created:<br/> 1) <a href="http://review.whamcloud.com/3203" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3203</a> - for 1.8 clients;<br/> 2) <a href="http://review.whamcloud.com/3202" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3202</a> - for 2.x servers.<br/> (<a href="http://morpheus.xyus.xyratex.com:8443/gerrit/#/c/120/" class="external-link" target="_blank" rel="nofollow noopener">http://morpheus.xyus.xyratex.com:8443/gerrit/#/c/120/</a> in Xyratex)<br/> 3) Landing History:<br/> Stage 1:<br/> <a href="http://review.whamcloud.com/3203" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3203</a> - was created for 1.8 clients<br/> <a href="http://review.whamcloud.com/3202" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3202</a> - was created for 2.x servers.<br/> Stage 2:<br/> Andreas requested to provide number of patches to reserve the flag value. So, next patches were added:<br/> <a href="http://review.whamcloud.com/#change,3722" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3722</a> (master) - landed<br/> <a href="http://review.whamcloud.com/#change,3725" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3725</a> (b2_2) - landed<br/> <a href="http://review.whamcloud.com/#change,3727" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/#change,3727</a> (b2_1) - approved. not landed yet.<br/> Stage 3 (current):<br/> <a href="http://review.whamcloud.com/3203" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3203</a> - Andreas don't want to have this patch, since without <a href="https://bugzilla.lustre.org/attachment.cgi?id=30902" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.lustre.org/attachment.cgi?id=30902</a> it has no sense. But for <a href="https://bugzilla.lustre.org/attachment.cgi?id=30902" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.lustre.org/attachment.cgi?id=30902</a> he see no benefit. Oleg has another vision and now it's up from WC to decide if they want to have these both patches in b1_8<br/> <a href="http://review.whamcloud.com/3202" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3202</a> - was rebased after landing 3722. Oleg think that we can't use this patch. I just discussed with him requirements. They are next:<br/> there is a 2.1 version where old policy is used. The patch for b1_8 should contain the ability to understand if the current server version support new policy or not. So, I fixed the situation "new client / new server". Now we have to fix "new client / old server".<br/> Open questions:<br/> 1) It's not clear if Andreas will agree with Oleg's recommendation (do they want to have <a href="https://bugzilla.lustre.org/attachment.cgi?id=30902" class="external-link" target="_blank" rel="nofollow noopener">https://bugzilla.lustre.org/attachment.cgi?id=30902</a>). </p> <p>Since <a href="http://review.whamcloud.com/3203" class="external-link" target="_blank" rel="nofollow noopener">http://review.whamcloud.com/3203</a> is abandoned, is there any reason to keep this ticket open?</p> <p>Close old bug.</p> Related LU-1137 LU-1770 Development Rank 1|hzvf7r: Rank (Obsolete) 6110