<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 02:23:32 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-9133] Use only MDS-local time when determining if directory is active</title>
                <link>https://jira.whamcloud.com/browse/LU-9133</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;When determining if a directory has recently been active, the client-provided timestamp in la_ctime is compared against the current time on the MDS. If the clocks are not synchronized on the client and the MDS, an incorrect elapsed time can be calculated. The comparison should instead be performed against the lr_contention_time in the resource, which is set by the MDS.&lt;/p&gt;

&lt;p&gt;Andreas Dilger provided detailed information about the required changes:&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;                &lt;span class=&quot;code-keyword&quot;&gt;if&lt;/span&gt; (!(child_bits &amp;amp; MDS_INODELOCK_UPDATE) &amp;amp;&amp;amp;
                      mdt_object_exists(child) &amp;amp;&amp;amp; !mdt_object_remote(child)) {
                        struct md_attr *ma = &amp;amp;info-&amp;gt;mti_attr;

                        ma-&amp;gt;ma_valid = 0;
                        ma-&amp;gt;ma_need = MA_INODE;
                        rc = mdt_attr_get_complex(info, child, ma);
                        &lt;span class=&quot;code-keyword&quot;&gt;if&lt;/span&gt; (unlikely(rc != 0))
                                GOTO(out_child, rc);

                        /* If the file has not been changed &lt;span class=&quot;code-keyword&quot;&gt;for&lt;/span&gt; some time, we
                         * &lt;span class=&quot;code-keyword&quot;&gt;return&lt;/span&gt; not only a LOOKUP lock, but also an UPDATE
                         * lock and &lt;span class=&quot;code-keyword&quot;&gt;this&lt;/span&gt; might save us RPC on later STAT. For
                         * directories, it also let negative dentry cache start
                         * working &lt;span class=&quot;code-keyword&quot;&gt;for&lt;/span&gt; &lt;span class=&quot;code-keyword&quot;&gt;this&lt;/span&gt; dir. */
                        &lt;span class=&quot;code-keyword&quot;&gt;if&lt;/span&gt; (ma-&amp;gt;ma_valid &amp;amp; MA_INODE &amp;amp;&amp;amp;
                            ma-&amp;gt;ma_attr.la_valid &amp;amp; LA_CTIME &amp;amp;&amp;amp;
                            info-&amp;gt;mti_mdt-&amp;gt;mdt_namespace-&amp;gt;ns_ctime_age_limit +
                                ma-&amp;gt;ma_attr.la_ctime &amp;lt; cfs_time_current_sec())
                                child_bits |= MDS_INODELOCK_UPDATE;
                }

&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;blockquote&gt;
&lt;p&gt;I think instead of looking up the inode and checking the client-supplied la_ctime on the directory against the MDS-local clock, this check should look up the resource and compare res-&amp;gt;lr_contention_time via ldlm_check_contention() which uses only the clock on the server. That would remove similar, but different code, and unify lock contention behaviour between the OST and MDT somewhat.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="43906">LU-9133</key>
            <summary>Use only MDS-local time when determining if directory is active</summary>
                <type id="4" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11310&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="4" iconUrl="https://jira.whamcloud.com/images/icons/priorities/minor.svg">Minor</priority>
                        <status id="1" iconUrl="https://jira.whamcloud.com/images/icons/statuses/open.png" description="The issue is open and ready for the assignee to start work on it.">Open</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="tappro">Mikhail Pershin</assignee>
                                    <reporter username="sguminsx">Steve Guminski</reporter>
                        <labels>
                    </labels>
                <created>Thu, 16 Feb 2017 19:17:34 +0000</created>
                <updated>Wed, 24 Jan 2024 19:43:39 +0000</updated>
                                            <version>Lustre 2.9.0</version>
                                    <fixVersion>Lustre 2.16.0</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>6</watches>
                                                                            <comments>
                            <comment id="187056" author="adilger" created="Sat, 4 Mar 2017 04:36:39 +0000"  >&lt;p&gt;From an email discussion between Steve and I:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;It appears that &lt;tt&gt;lhc&lt;/tt&gt; is an &lt;tt&gt;mdt_lock_handle()&lt;/tt&gt; for the child directory entry, and &lt;tt&gt;lhp&lt;/tt&gt; is a lock handle for the parent directory. You can turn these into DLM locks via &lt;tt&gt;lock = ldlm_handle2lock(&amp;amp;lhc-&amp;gt;mlh_reg_lh);&lt;/tt&gt;&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;Thanks for the info, Andreas. I&apos;ve been trying to follow the instructions you provided, but &lt;tt&gt;ldlm_handle2lock(&amp;amp;lhc-&amp;gt;mlh_reg_lh)&lt;/tt&gt; seems to always be returning &lt;tt&gt;NULL&lt;/tt&gt;. Shortly before the code that I&apos;m replacing, there is a call to &lt;tt&gt;mdt_lock_handle_init(lhc)&lt;/tt&gt;. This function zeroes out the &lt;tt&gt;lhc-&amp;gt;mlh_reg_lh.cookie&lt;/tt&gt;, which seems to be needed by &lt;tt&gt;ldlm_handle2lock()&lt;/tt&gt; to actually locate the lock.&lt;/p&gt;

&lt;p&gt;I see that a few lines later, &lt;tt&gt;mdt_object_lock()&lt;/tt&gt; is called for the child. Is this where the handle&apos;s cookie gets set? If so, then there&apos;s a conflict: I need to know the child_bits before calling &lt;tt&gt;mdt_object_lock()&lt;/tt&gt;, but I can&apos;t get the child_bits until I&apos;ve called &lt;tt&gt;mdt_object_lock()&lt;/tt&gt; to get the lock and retrieve the contention time.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;According to the existing code, which is using &lt;tt&gt;child&lt;/tt&gt; and the timestamps thereon, it makes sense to check the &lt;tt&gt;lhc&lt;/tt&gt; data rather than &lt;tt&gt;lhp&lt;/tt&gt;.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;If I instead try to locate the lock for the parent (&lt;tt&gt;lhp&lt;/tt&gt;), I get a successful result from &lt;tt&gt;ldlm_handle2lock()&lt;/tt&gt;. I&apos;m not sure if the parent lock is actually useful, however.&lt;/p&gt;

&lt;p&gt;I also tried to find the &lt;tt&gt;ldlm_resource&lt;/tt&gt; to directly compare the &lt;tt&gt;res-&amp;gt;lr_contention_time&lt;/tt&gt;.  I tried to get the ldlm_res_id from the child_fid using &lt;tt&gt;fid_build_reg_res_name()&lt;/tt&gt;, and then called &lt;tt&gt;ldlm_resource_get()&lt;/tt&gt; to find the resource.  However, the latter function returned &lt;tt&gt;NULL&lt;/tt&gt;.  Is this approach incorrect?&lt;/p&gt;</comment>
                            <comment id="187057" author="adilger" created="Sat, 4 Mar 2017 04:41:38 +0000"  >&lt;blockquote&gt;
&lt;p&gt;I also tried to find the &lt;tt&gt;ldlm_resource&lt;/tt&gt; to directly compare the &lt;tt&gt;res-&amp;gt;lr_contention_time&lt;/tt&gt;.  I tried to get the ldlm_res_id from the child_fid using &lt;tt&gt;fid_build_reg_res_name()&lt;/tt&gt;, and then called &lt;tt&gt;ldlm_resource_get()&lt;/tt&gt; to find the resource.  However, the latter function returned &lt;tt&gt;NULL&lt;/tt&gt;.  Is this approach incorrect?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;That would have been my next choice.  It is possible the LDLM namespace (&lt;tt&gt;info&amp;#45;&amp;gt;mti&amp;#95;mdt&amp;#45;&amp;gt;mdt&amp;#95;namespace&lt;/tt&gt;) or resource type (&lt;tt&gt;LDLM_IBITS&lt;/tt&gt;) was not correct?  Looking down the callpath, it looks like &lt;tt&gt;mdt_object_local_lock()&lt;/tt&gt; is doing the same. I do agree there is a chicken-and-egg problem here, so this may not be as easy as I thought.&lt;/p&gt;

&lt;p&gt;On the plus side, there is work being done by &lt;a href=&quot;https://jira.whamcloud.com/secure/ViewProfile.jspa?name=tappro&quot; class=&quot;user-hover&quot; rel=&quot;tappro&quot;&gt;tappro&lt;/a&gt; for Data-on-MDT in &lt;a href=&quot;https://review.whamcloud.com/23010&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/23010&lt;/a&gt; to allow speculatively adding/removing lock bits to a lock, which may allow us to do the lookup of the child lock/resource with the UPDATE bit optional, and then not grant that bit in the DLM if there is resource contention.  With that said, I think it makes sense to shelve this change until Mike&apos;s work lands.&lt;/p&gt;</comment>
                            <comment id="187356" author="tappro" created="Tue, 7 Mar 2017 18:07:27 +0000"  >&lt;p&gt;Andreas, do you mean that &apos;try_bits&apos; patch: &lt;a href=&quot;https://review.whamcloud.com/#/c/25262/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/#/c/25262/&lt;/a&gt;&#160;? Or whole combination with lock convert? The first one os ready for landing, just review is required.&lt;/p&gt;</comment>
                            <comment id="187522" author="adilger" created="Wed, 8 Mar 2017 20:29:06 +0000"  >&lt;p&gt;I guess I meant the patch &lt;a href=&quot;https://review.whamcloud.com/25262&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/25262&lt;/a&gt; (&quot;&lt;a href=&quot;https://jira.whamcloud.com/browse/LU-9184&quot; title=&quot;early patches for Data-on-MDT support&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-9184&quot;&gt;&lt;del&gt;LU-9184&lt;/del&gt;&lt;/a&gt; ldlm: selective IBITS lock trying&quot;).  The issue here is that we can&apos;t check the contention time on the DLM resource before getting the lock, but we don&apos;t know if we want the UPDATE bit until after checking the lock contention...&lt;/p&gt;

&lt;p&gt;My thought to fix this properly is that the caller will always speculatively try to get the UPDATE bit, but the DLM code (or MDS callback?) can drop the UPDATE bit if the lock is recently contended.  That keeps the info about lock contention internal to the DLM instead of the caller, and avoids the chicken-and-egg problem.&lt;/p&gt;</comment>
                            <comment id="187697" author="tappro" created="Thu, 9 Mar 2017 19:25:27 +0000"  >&lt;p&gt;I see, so for this particular ticket &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-9133&quot; title=&quot;Use only MDS-local time when determining if directory is active&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-9133&quot;&gt;LU-9133&lt;/a&gt; the only patch for selective lock bits is needed, not whole DoM work.&lt;/p&gt;</comment>
                            <comment id="232460" author="adilger" created="Wed, 22 Aug 2018 21:28:30 +0000"  >&lt;p&gt;Mike, is there some chance that this could be fixed for 2.12 still?  We still get occasional complaints about slow operations when the system clocks are not in sync.&lt;/p&gt;</comment>
                            <comment id="232472" author="tappro" created="Thu, 23 Aug 2018 06:38:50 +0000"  >&lt;p&gt;Andreas, I see that contention mechanism is not being used with IBITS locks. This must be done first to update lr_contention_time regularly. The simplest way would be to count each conflicting lock as contended one, maybe you have an better idea about contention for IBITS locks?&lt;/p&gt;</comment>
                            <comment id="232702" author="adilger" created="Tue, 28 Aug 2018 21:08:01 +0000"  >&lt;p&gt;Mike, I think that is exactly what lr_contention_time is for - to track how recently the resource was contended by multiple clients, so that the server can make better decisions than just ping-pong of the lock.  This should apply to conflicting IBITS locks as well, if it doesn&apos;t yet.&lt;/p&gt;</comment>
                            <comment id="234101" author="tappro" created="Fri, 28 Sep 2018 12:52:48 +0000"  >&lt;p&gt;Andreas, I am not sure that lr_contention_time can replace ctime check in that code, lr_contention_time is updated when amount of contended locks are bigger than ns_contended_locks variable which is 32, therefore this time is updated much rarely than ctime. Is that what we need?&lt;/p&gt;</comment>
                            <comment id="234120" author="adilger" created="Fri, 28 Sep 2018 22:13:27 +0000"  >&lt;p&gt;Hmm. I thought it was updated on the most recent conflict, but I haven&apos;t looked at it recently. If there is some other way to determine if the directory has been idle recently, I&apos;d be fine with a different solution. Using timestamps from the MDS is best, since client and MDS clocks may be out of sync. Also, clocks between clients may be different, so using the MDS clock is best. &lt;/p&gt;</comment>
                            <comment id="234862" author="adilger" created="Fri, 12 Oct 2018 23:43:18 +0000"  >&lt;p&gt;Quentin, is there a chance you could look into this issue?&#160; It is somewhat related to your work on &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11199&quot; title=&quot;mdsrate open() performance degradation&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11199&quot;&gt;&lt;del&gt;LU-11199&lt;/del&gt;&lt;/a&gt; &quot;mdt: Attempt lookup lock on open&quot;&lt;/p&gt;</comment>
                            <comment id="234913" author="bougetq" created="Mon, 15 Oct 2018 07:57:05 +0000"  >&lt;p&gt;@Andreas, I am pretty sure you meant someone else... Maybe Patrick Farell?&lt;/p&gt;</comment>
                            <comment id="234926" author="adilger" created="Mon, 15 Oct 2018 15:19:49 +0000"  >&lt;p&gt;Sorry, I meant Dominique&apos;s work in patch &lt;a href=&quot;https://review.whamcloud.com/30880&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/30880&lt;/a&gt; &quot;&lt;tt&gt;U-10235 mdt: mdt_create: check EEXIST without lock&lt;/tt&gt;&quot;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10120">
                    <name>Blocker</name>
                                                                <inwardlinks description="is blocked by">
                                        <issuelink>
            <issuekey id="53433">LU-11435</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                            <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzz46n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>