<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 01:25:36 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-2484] md_stats not used or used incorrectly</title>
                <link>https://jira.whamcloud.com/browse/LU-2484</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;The stats maintained by the EXP_MD_COUNTER_INCREMENT in the md_ops member wrappers and stored in struct obd_device&apos;s md_stats are not used.  The classes that implement the the md_ops interface (lmv and mdc) do not allocate the them and the classes that allocate them (mgs and mdt) do not implement md_ops.  (This is not a terrible issue but it does add to the internal and external cognitive crud of lustre proc.)&lt;/p&gt;

&lt;p&gt;It should be easy to add these stats to lmv and mds but the value of doing so seems low.  md_ops contains several groups of low-level function whose counts would be highly correlated and there are already counters in llite for the high level metadata operations.&lt;/p&gt;

&lt;p&gt;If they are to be enabled anywhere then EXP_MD_COUNTER_INCREMENT() should check exp-&amp;gt;exp_obd-&amp;gt;md_stats rather than exp-&amp;gt;exp_obd-&amp;gt;obd_stats; lprocfs_alloc_md_stats() should check that obd-&amp;gt;obd_type-&amp;gt;typ_md_ops is not NULL; and exp_md_stats should be used somewhere or removed.&lt;/p&gt;</description>
                <environment></environment>
        <key id="16911">LU-2484</key>
            <summary>md_stats not used or used incorrectly</summary>
                <type id="4" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11310&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="4" iconUrl="https://jira.whamcloud.com/images/icons/priorities/minor.svg">Minor</priority>
                        <status id="5" iconUrl="https://jira.whamcloud.com/images/icons/statuses/resolved.png" description="A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.">Resolved</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="1">Fixed</resolution>
                                        <assignee username="jhammond">John Hammond</assignee>
                                    <reporter username="jhammond">John Hammond</reporter>
                        <labels>
                            <label>patch</label>
                            <label>procfs</label>
                    </labels>
                <created>Wed, 12 Dec 2012 12:25:11 +0000</created>
                <updated>Tue, 15 Oct 2013 22:01:37 +0000</updated>
                            <resolved>Tue, 15 Oct 2013 22:01:37 +0000</resolved>
                                    <version>Lustre 2.4.0</version>
                                    <fixVersion>Lustre 2.4.0</fixVersion>
                    <fixVersion>Lustre 2.5.0</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>4</watches>
                                                                            <comments>
                            <comment id="49137" author="adilger" created="Wed, 12 Dec 2012 12:35:50 +0000"  >&lt;p&gt;John, with the advent of multiple MDTs, the stats per MDC will no longer be in lockstep with those at the llite/LMV level.  It would be nice to have the per-MDC metadata stats available on the client (e.g. for &lt;a href=&quot;http://review.whamcloud.com/4403&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/4403&lt;/a&gt;) if you think they provide some value.&lt;/p&gt;

&lt;p&gt;In any case, it definitely seems like there is some cleanup to do, and as always I&apos;m appreciative of your efforts.&lt;/p&gt;</comment>
                            <comment id="49138" author="jhammond" created="Wed, 12 Dec 2012 12:41:44 +0000"  >&lt;p&gt;See &lt;a href=&quot;http://review.whamcloud.com/4813&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/4813&lt;/a&gt; for a patch to remove md_stats from mgs and add some error handling to mgs proc initialization.&lt;/p&gt;</comment>
                            <comment id="49139" author="jhammond" created="Wed, 12 Dec 2012 12:50:59 +0000"  >&lt;p&gt;&amp;gt; John, with the advent of multiple MDTs, the stats per MDC will no longer be in lockstep with those at the llite/LMV level. It would be nice to have the per-MDC metadata stats available on the client&lt;/p&gt;

&lt;p&gt;That seems fine to me.  I had planned to do this in three parts:&lt;br/&gt;
 1) Remove from mgs (see above)&lt;br/&gt;
 2) Remove from mdt.&lt;br/&gt;
 3) Remove common functions and members.&lt;/p&gt;

&lt;p&gt;I&apos;ll go ahead with 1 and 2.  Rather than just turning these on for lmv and mdc, I think the calls to EXP_MD_COUNTER_INCREMENT() should first be pruned.  I don&apos;t believe that knowing the call count of mdc_init_ea_size() assists anyone in mentally modeling the behavior of lustre.&lt;/p&gt;</comment>
                            <comment id="49144" author="jhammond" created="Wed, 12 Dec 2012 13:38:34 +0000"  >&lt;p&gt;See &lt;a href=&quot;http://review.whamcloud.com/4815&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/4815&lt;/a&gt; for the mdt patch.&lt;/p&gt;</comment>
                            <comment id="49209" author="jhammond" created="Thu, 13 Dec 2012 16:48:59 +0000"  >&lt;p&gt;And &lt;a href=&quot;http://review.whamcloud.com/4827&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://review.whamcloud.com/4827&lt;/a&gt; for the mdc patch.&lt;/p&gt;</comment>
                            <comment id="51150" author="jhammond" created="Thu, 24 Jan 2013 17:21:52 +0000"  >&lt;p&gt;I don&apos;t think these stats should be added to MDCs or any other class. Before I say why here&apos;s what they will look like:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;~# cat /proc/fs/lustre/mdc/lustre-MDT0000-mdc-ffff88020176ec00/md_stats
snapshot_time             1359065590.107282 secs.usecs
getstatus                 1 samples [reqs]
change_cbdata             2246 samples [reqs]
find_cbdata               2413 samples [reqs]
close                     157 samples [reqs]
create                    1667 samples [reqs]
enqueue                   12 samples [reqs]
getattr                   42 samples [reqs]
getattr_name              5 samples [reqs]
intent_lock               13239 samples [reqs]
link                      1 samples [reqs]
rename                    17 samples [reqs]
setattr                   113 samples [reqs]
readpage                  17 samples [reqs]
unlink                    537 samples [reqs]
getxattr                  155 samples [reqs]
init_ea_size              4 samples [reqs]
get_lustre_md             2857 samples [reqs]
free_lustre_md            2752 samples [reqs]
set_open_replay_data      157 samples [reqs]
clear_open_replay_data    157 samples [reqs]
set_lock_data             9897 samples [reqs]
lock_match                2306 samples [reqs]
intent_getattr_async      33 samples [reqs]
revalidate_lock           3375 samples [reqs]
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;Now imagine someone reading this and saying &quot;Whenever my jobs run the clients show a lot of change_cbdata and revalidate_lock reqs. Is that bad?&quot;&lt;/p&gt;

&lt;p&gt;What would you say to them? I would say that I&apos;m sorry I submitted a patch to enable these stats.&lt;/p&gt;

&lt;p&gt;The point is that these stats are counts of calls, and not of high-level metadata operations that help admins and users model their filesystem.&lt;/p&gt;</comment>
                            <comment id="58914" author="keith" created="Mon, 20 May 2013 17:59:57 +0000"  >&lt;p&gt;Have we thought about moving part of this stat data to a perf tracepoints sort of implementation with the idea to keep admin friendly items in /proc and developer data elsewhere? &lt;/p&gt;
</comment>
                            <comment id="59015" author="jhammond" created="Tue, 21 May 2013 21:58:28 +0000"  >&lt;p&gt;Keith, this data is well used by admins. The big sites support a healthy cottage industry of lustre monitoring tools.&lt;/p&gt;</comment>
                            <comment id="65591" author="jhammond" created="Tue, 3 Sep 2013 11:00:23 +0000"  >&lt;p&gt;Patch landed to master.&lt;/p&gt;</comment>
                            <comment id="69036" author="jlevi" created="Tue, 15 Oct 2013 22:01:37 +0000"  >&lt;p&gt;Added 2.5.0 FixVersion&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                            <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzvdpb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5826</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>