<!-- 
RSS generated by JIRA (9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c) at Sat Feb 10 02:43:36 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>Whamcloud Community JIRA</title>
    <link>https://jira.whamcloud.com</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.4.14</version>
        <build-number>940014</build-number>
        <build-date>05-12-2023</build-date>
    </build-info>


<item>
            <title>[LU-11407] Improve stats data</title>
                <link>https://jira.whamcloud.com/browse/LU-11407</link>
                <project id="10000" key="LU">Lustre</project>
                    <description>&lt;p&gt;It would be useful to store and report the &quot;job start&quot; time for the JobStats.  Currently we show in the &lt;tt&gt;obdfilter.&amp;#42;.job_stats&lt;/tt&gt; file:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;- job_id:          mythbackend.0
  snapshot_time:   1537384753
  read_bytes:      { samples:         321, unit: bytes, min:    4096, max: 4194304, sum:      1025404928 }
  write_bytes:     { samples:       12656, unit: bytes, min:   22028, max:  919476, sum:      5413800656 }
  sync:            { samples:       11168, unit:  reqs }
  statfs:          { samples:       31249, unit:  reqs }
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;but this doesn&apos;t tell us anything about when this job started, so we can&apos;t find the throughput or IOPS rates.  It should be simple to store the first time this job reported IO so that we can have some idea about the rate.&lt;/p&gt;

&lt;p&gt;A further enhancement would be to store the full &lt;tt&gt;brw_stats&lt;/tt&gt; into the &lt;tt&gt;job_stats&lt;/tt&gt; file, but that is a more complex change.&lt;/p&gt;</description>
                <environment></environment>
        <key id="53365">LU-11407</key>
            <summary>Improve stats data</summary>
                <type id="4" iconUrl="https://jira.whamcloud.com/secure/viewavatar?size=xsmall&amp;avatarId=11310&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="4" iconUrl="https://jira.whamcloud.com/images/icons/priorities/minor.svg">Minor</priority>
                        <status id="5" iconUrl="https://jira.whamcloud.com/images/icons/statuses/resolved.png" description="A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.">Resolved</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="1">Fixed</resolution>
                                        <assignee username="adilger">Andreas Dilger</assignee>
                                    <reporter username="adilger">Andreas Dilger</reporter>
                        <labels>
                    </labels>
                <created>Wed, 19 Sep 2018 21:02:58 +0000</created>
                <updated>Tue, 1 Aug 2023 12:58:42 +0000</updated>
                            <resolved>Tue, 26 Jul 2022 13:55:43 +0000</resolved>
                                                    <fixVersion>Lustre 2.16.0</fixVersion>
                    <fixVersion>Lustre 2.15.2</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>7</watches>
                                                                            <comments>
                            <comment id="233774" author="adilger" created="Wed, 19 Sep 2018 22:01:07 +0000"  >&lt;p&gt;Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/33201&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33201&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; obdclass: include start time in job_stats &lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 9a40d022971d9078175c1f0ba1a399a03cfbc4c7&lt;/p&gt;
</comment>
                            <comment id="234248" author="gerrit" created="Tue, 2 Oct 2018 21:57:36 +0000"  >&lt;p&gt;&lt;del&gt;Andreas Dilger (adilger@whamcloud.com) uploaded a new patch:&lt;/del&gt; &lt;a href=&quot;https://review.whamcloud.com/33270&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33270&lt;/a&gt;&lt;br/&gt;
&lt;del&gt;Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; obdclass: add start time to stats files&lt;/del&gt;&lt;br/&gt;
&lt;del&gt;Project: fs/lustre-release&lt;/del&gt;&lt;br/&gt;
&lt;del&gt;Branch: master&lt;/del&gt;&lt;br/&gt;
&lt;del&gt;Current Patch Set: 1&lt;/del&gt;&lt;br/&gt;
&lt;del&gt;Commit: b26844d1b6d593810c04bb4663bb578f77ec0b86&lt;/del&gt;&lt;/p&gt;</comment>
                            <comment id="234561" author="joe.grund" created="Mon, 8 Oct 2018 14:38:23 +0000"  >&lt;p&gt;What release(s) is this enhancement planning to land in?&lt;/p&gt;</comment>
                            <comment id="234562" author="joe.grund" created="Mon, 8 Oct 2018 14:38:49 +0000"  >&lt;p&gt;Is there a sample of how the new output will look?&lt;/p&gt;</comment>
                            <comment id="234621" author="lixi_wc" created="Tue, 9 Oct 2018 11:39:53 +0000"  >&lt;p&gt;We might need to create seperate tickets, but we have some requirements for the stats improvement:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Printing the $time_interval, $sum / $time_interval, $sumsquare / $counter etc directly. Thus the collector doesn&apos;t need to parse the content and calculate the values.&lt;/li&gt;
	&lt;li&gt;In order to get some meaningful distribution during a given time interval, e.g. the I/O size distribution (i.e. how many percentage each I/O size has) during the data collecting interval, we need to A) read the /proc entry and then B) clear the data counters by writing to the /proc entry. This works fine, but still has problem, because there is still time interval between step A) and B), at least in theory. It would be nice that Lustre can provide some kind of parameter or option. If the option is enabled, reading the data from /proc would clear the counters into zero at the same time.&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="234627" author="simmonsja" created="Tue, 9 Oct 2018 14:09:14 +0000"  >&lt;p&gt;Some time back a patch for the kernel code was pushed for 1) and it was rejected. Now if you really want it we could make &quot;lctl get_param **.*stats&quot; a wrapper around a function in liblustreapi that does these calculations for you.&#160;&lt;/p&gt;</comment>
                            <comment id="234830" author="adilger" created="Fri, 12 Oct 2018 15:01:17 +0000"  >&lt;p&gt;I was hoping to include it in 2.12 as a very minor enhancement, but if there is a significant issue affecting the parser then I could wait. I&apos;ve also changed the job stats to have a sec.nsec timestamp to make it consistent with other stats. &lt;/p&gt;

&lt;p&gt;I&apos;ve refreshed the patch (will push soon) to include a field &lt;tt&gt;elapsed_time:&lt;/tt&gt; which is the difference between the start time and current time, so Li Xi&apos;s parser doesn&apos;t need to do that. Doing all of the division in the kernel is problematic because the kernel does not support floating-point math. &lt;/p&gt;

&lt;p&gt;The output for any stat that has &lt;tt&gt;snapshot_time:&lt;/tt&gt; at the start will get two additional lines:&lt;/p&gt;
&lt;div class=&quot;preformatted panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;preformattedContent panelContent&quot;&gt;
&lt;pre&gt;snapshot_time: 123456789.123456789 (secs.nsec)
start_time:    123456678.012345678
elapsed_time:       1111.111111111
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
</comment>
                            <comment id="234832" author="joe.grund" created="Fri, 12 Oct 2018 15:04:34 +0000"  >&lt;p&gt;No issue on my end, just want to know where I need to target.&lt;/p&gt;</comment>
                            <comment id="235149" author="lixi_wc" created="Fri, 19 Oct 2018 09:28:15 +0000"  >&lt;p&gt;I don&apos;t think floating-point math is necessary, since 64 bit interger should be enough for most of the collectors. A rate with high precision doesn&apos;t help too much for analysis.&lt;/p&gt;

&lt;p&gt;Anyway, the elapsed_time helps a lot. Thanks.&lt;/p&gt;</comment>
                            <comment id="264342" author="gerrit" created="Sun, 1 Mar 2020 09:35:31 +0000"  >&lt;p&gt;Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/37764&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/37764&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; tgt: cleanup job_stats output printing&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: cf1e6bb6403ff3114976c3c07e1aa65ab9230db3&lt;/p&gt;</comment>
                            <comment id="316624" author="gerrit" created="Wed, 27 Oct 2021 00:35:32 +0000"  >&lt;p&gt;&quot;Oleg Drokin &amp;lt;green@whamcloud.com&amp;gt;&quot; merged in patch &lt;a href=&quot;https://review.whamcloud.com/33201/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/33201/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; obdclass: add start time to stats files&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: ea2cd3af7bfabfa6876727ee44495f4c331bea8e&lt;/p&gt;</comment>
                            <comment id="341534" author="gerrit" created="Tue, 26 Jul 2022 04:53:29 +0000"  >&lt;p&gt;&quot;Oleg Drokin &amp;lt;green@whamcloud.com&amp;gt;&quot; merged in patch &lt;a href=&quot;https://review.whamcloud.com/37764/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/37764/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; tgt: cleanup job_stats output printing&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 338381574b643da064e90e44d75be85d1be3a93c&lt;/p&gt;</comment>
                            <comment id="341597" author="pjones" created="Tue, 26 Jul 2022 13:55:43 +0000"  >&lt;p&gt;Landed for 2.16&lt;/p&gt;</comment>
                            <comment id="346534" author="gerrit" created="Tue, 13 Sep 2022 18:31:49 +0000"  >&lt;p&gt;&quot;Jian Yu &amp;lt;yujian@whamcloud.com&amp;gt;&quot; uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/48539&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/48539&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; tgt: cleanup job_stats output printing&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: b2_15&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: 8eea90f503a35942c8af25520d6485827f9370f3&lt;/p&gt;</comment>
                            <comment id="347916" author="gerrit" created="Mon, 26 Sep 2022 16:57:38 +0000"  >&lt;p&gt;&quot;Oleg Drokin &amp;lt;green@whamcloud.com&amp;gt;&quot; merged in patch &lt;a href=&quot;https://review.whamcloud.com/48539/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/48539/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; tgt: cleanup job_stats output printing&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: b2_15&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 56c0d6316fbf29eac019f5a7c823199592027b25&lt;/p&gt;</comment>
                            <comment id="370455" author="gerrit" created="Tue, 25 Apr 2023 06:36:17 +0000"  >&lt;p&gt;&quot;Andreas Dilger &amp;lt;adilger@whamcloud.com&amp;gt;&quot; uploaded a new patch: &lt;a href=&quot;https://review.whamcloud.com/c/fs/lustre-release/+/50734&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/c/fs/lustre-release/+/50734&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; obdclass: init osc.*.rpc_stats start_time&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: 1&lt;br/&gt;
Commit: f8511d3f4ed5a7a2b426383f036208ec64de1cf5&lt;/p&gt;</comment>
                            <comment id="371010" author="gerrit" created="Mon, 1 May 2023 04:12:30 +0000"  >&lt;p&gt;&quot;Oleg Drokin &amp;lt;green@whamcloud.com&amp;gt;&quot; merged in patch &lt;a href=&quot;https://review.whamcloud.com/c/fs/lustre-release/+/50734/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://review.whamcloud.com/c/fs/lustre-release/+/50734/&lt;/a&gt;&lt;br/&gt;
Subject: &lt;a href=&quot;https://jira.whamcloud.com/browse/LU-11407&quot; title=&quot;Improve stats data&quot; class=&quot;issue-link&quot; data-issue-key=&quot;LU-11407&quot;&gt;&lt;del&gt;LU-11407&lt;/del&gt;&lt;/a&gt; obdclass: init osc.*.rpc_stats start_time&lt;br/&gt;
Project: fs/lustre-release&lt;br/&gt;
Branch: master&lt;br/&gt;
Current Patch Set: &lt;br/&gt;
Commit: 0176531449899c30ebdeaf372464fd0685ca3645&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Related</name>
                                            <outwardlinks description="is related to ">
                                        <issuelink>
            <issuekey id="59312">LU-13597</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="72739">LU-16231</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="70159">LU-15826</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="57775">LU-13123</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="57177">LU-12872</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="74683">LU-16555</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="74846">LU-16599</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                            <customfield id="customfield_10890" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10390" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i002r3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10090" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                </customfields>
    </item>
</channel>
</rss>