https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13490 Lustre <p>Parallel readahead introcued after <a href="https://jira.whamcloud.com/browse/LU-12043" title="improve Lustre single thread read performances" class="issue-link" data-issue-key="LU-12043"><del>LU-12043</del></a> (commit c279167) and kernel threads does readahead in parallel, but there is a regression that broek read stats in jobstats.<br/> Here is a reproducer.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@mgs ~]# lctl conf_param vLustre.sys.jobid_var=procname_uid [root@client ~]# ior -w -t 1m -b 1g -e -o /vLustre/out/file -k [root@client ~]# echo 3 &gt; /proc/sys/vm/drop_caches [root@client ~]# ior -r -t 1m -b 1g -e -o /vLustre/out/file -k [root@oss1 ~]# lctl get_param obdfilter.*.job_stats obdfilter.vLustre-OST0000.job_stats= job_stats: - job_id: ior.0 snapshot_time: 1588138284 read_bytes: { samples: 16, unit: bytes, min: 1048576, max: 4194304, sum: 62914560 } write_bytes: { samples: 256, unit: bytes, min: 4194304, max: 4194304, sum: 1073741824 } getattr: { samples: 0, unit: reqs } setattr: { samples: 0, unit: reqs } punch: { samples: 0, unit: reqs } sync: { samples: 1, unit: reqs } destroy: { samples: 0, unit: reqs } create: { samples: 0, unit: reqs } statfs: { samples: 0, unit: reqs } get_info: { samples: 0, unit: reqs } set_info: { samples: 0, unit: reqs } quotactl: { samples: 0, unit: reqs } - job_id: kworker/u4:1.0 snapshot_time: 1588138285 read_bytes: { samples: 135, unit: bytes, min: 4194304, max: 4194304, sum: 566231040 } write_bytes: { samples: 0, unit: bytes, min: 0, max: 0, sum: 0 } getattr: { samples: 0, unit: reqs } setattr: { samples: 0, unit: reqs } punch: { samples: 0, unit: reqs } sync: { samples: 0, unit: reqs } destroy: { samples: 0, unit: reqs } create: { samples: 0, unit: reqs } statfs: { samples: 0, unit: reqs } get_info: { samples: 0, unit: reqs } set_info: { samples: 0, unit: reqs } quotactl: { samples: 0, unit: reqs } - job_id: kworker/u4:3.0 snapshot_time: 1588138284 read_bytes: { samples: 106, unit: bytes, min: 4194304, max: 4194304, sum: 444596224 } write_bytes: { samples: 0, unit: bytes, min: 0, max: 0, sum: 0 } getattr: { samples: 0, unit: reqs } setattr: { samples: 0, unit: reqs } punch: { samples: 0, unit: reqs } sync: { samples: 0, unit: reqs } destroy: { samples: 0, unit: reqs } create: { samples: 0, unit: reqs } statfs: { samples: 0, unit: reqs } get_info: { samples: 0, unit: reqs } set_info: { samples: 0, unit: reqs } quotactl: { samples: 0, unit: reqs } </pre> </div></div> <p>it's bad idea of tracking read stats per kernel thread rathar than real application pid. it won't be able to see read stats per job id.</p> LU-13490

readahead thread breaks read stats in jobstats

Bug Major Resolved Fixed Wang Shilong Shuichi Ihara Wed, 29 Apr 2020 05:34:57 +0000 Thu, 23 Sep 2021 17:19:11 +0000 Thu, 14 May 2020 14:50:43 +0000 Lustre 2.14.0 Lustre 2.14.0 0 9 <p>There is an exception table for jobid that skips specific thread names, but I don't think it can work in this case. Maybe it would be better to check if the thread has <tt>PF_KERNTHREAD</tt> set? Also, I think it is possible to cache the jobid in <tt>struct ll_jnode_info</tt>, so if that was set then it would report the correct jobid to the OSS. </p> <p>Shilong</p> <p>Could you please advise?</p> <p>Thanks</p> <p>Peter</p> <p>One of possible way to solve the issue could be we pass original task_struct to job_id, and use passed task_struct to pass jobid information which should fix this problem.</p> <p>Wang Shilong (wshilong@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/38426" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38426</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13490" title="readahead thread breaks read stats in jobstats" class="issue-link" data-issue-key="LU-13490"><del>LU-13490</del></a> lustre: fix to make jobstats work for async ra<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: cfdc2b04d3d89a1629e5df1dde882af34d18e4aa</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/38426/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/38426/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13490" title="readahead thread breaks read stats in jobstats" class="issue-link" data-issue-key="LU-13490"><del>LU-13490</del></a> lustre: fix to make jobstats work for async ra<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: da8972322134ae5741e7176312fca1f980c0f69a</p> <p>Landed for 2.14</p> Related Development Rank 1|i00z27: Rank (Obsolete) 9223372036854775807 Severity