https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-7880 Lustre <p>In order to facilitate transfer of OST and MDT performance statistics for userspace applications, such as global NRS scheduling, SCR checkpoint scheduling, QOS and allocation decisions on the MDS, etc, it is useful to transport them via <tt>obd_statfs</tt> to the clients.</p> <p>The statistics should include &lt;peak, decaying average of current&gt; &lt;IOPS read, IOPS write, KiB/s read, KiB/s write&gt;.</p> <p>The OSS and MDS already collect these statistics for presentation via <tt>/proc</tt> and it should be possible to include them into <tt>struct obd_statfs</tt> in newly-added fields at the end of the struct.</p> <p>The stats should be fetched and printed with <tt>lfs df --stats</tt> command for all targets, but not necessarily for regular <tt>statfs()</tt> requests. With <a href="https://jira.whamcloud.com/browse/LU-10018" title="MDT as a statfs proxy" class="issue-link" data-issue-key="LU-10018"><del>LU-10018</del></a> "<tt>MDT as a statfs() proxy</tt>", the <tt>MDT_STATFS</tt> request now has an <tt>mdt_body</tt> in the request which can be used to request different behaviour for the RPC.</p> LU-7880

add performance statistics to obd_statfs

Improvement Major Open Unresolved WC Triage Andreas Dilger medium Tue, 15 Mar 2016 02:41:14 +0000 Wed, 4 Oct 2023 15:21:21 +0000 0 8 <p>do I understand correctly, that OFD should track performance on its own? something like a separate thread (or timer-driven callback) collecting stats from OSD and maintaining a history of average/peak throughput, RPC rate?</p> <p>AFAIU, we don't track average for last few seconds, just average since start or reset.</p> <p> </p> <p>We already track stats on the OST and MDT for RPCs, read/write calls with min/max duration, read_bytes/write_bytes with sums. It should be fairly straight forward to use the existing stats counters to generate peak performance and decaying average performance, either directly or by doing simple delta calculations when statfs is called (eg. save the last time and last stats and do a simple rate calculation over the past minute or whatever). The MDS is already calling statfs in the background every 5s so that is often enough to keep this updated. </p> <p>I suppose you can get instantaneous rates for the last 5 seconds if you only record the stats when called by the MDT. I think 60-second averages are more useful so we don't have to poll statfs so often; I suppose we could only record the stats if the timestamp of the last record is greater than 60 seconds, so we would effectively have 60-second epochs.</p> Related LU-10018 LU-10070 LU-9 LU-10158 Development Rank 1|hzy4hj: Rank (Obsolete) 9223372036854775807