https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-13309 Lustre <p>A few trivial patches avoiding OSS CPU bottleneck with NVME storage will be uploaded shortly. </p> LU-13309

performance optimizations for brw

Improvement Minor Resolved Fixed Andrew Perepechko Andrew Perepechko patch performance Fri, 28 Feb 2020 13:16:42 +0000 Mon, 7 Feb 2022 14:06:28 +0000 Tue, 11 Jan 2022 14:16:14 +0000 Lustre 2.15.0 0 5 <p>Andrew Perepechko (c17827@cray.com) uploaded a new patch: <a href="https://review.whamcloud.com/37758" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37758</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13309" title="performance optimizations for brw" class="issue-link" data-issue-key="LU-13309"><del>LU-13309</del></a> osd-ldiskfs: remove per-page object_get/put in brw<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8524ffd75fa443d02c9ce74e9e998c02d8e55b72</p> <p>Andrew, what is particular workload you've seen this high CPU usages?</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=sihara" class="user-hover" rel="sihara">sihara</a>, it's an IOR IOPS test, i.e. random read 4 KiB chunk from a 16 GiB file, single OST. </p> <p>OSS configuration:<br/> CPU based on AMD Rome, NVME storage.</p> <p>Client configuration:<br/> 48 clients, 8 processes per node.</p> <p>Andrew Perepechko (c17827@cray.com) uploaded a new patch: <a href="https://review.whamcloud.com/37786" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37786</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13309" title="performance optimizations for brw" class="issue-link" data-issue-key="LU-13309"><del>LU-13309</del></a> osd-ldiskfs: speedup osd_bufs_get/put<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 7138a24a2362b8fd021e5fc000e671d309816534</p> <p>One more patch, brw_stats related, will be uploaded soon.</p> <p>The IOPS test shows ~590000 IOPS -&gt; ~620000 IOPS improvement with the three patches.</p> <p>Either with or without these three patches CPU is the bottleneck, 100% CPU load.</p> <p>The CPU profile shows that the load is spread between various CPU consumers, the most significant of which is memset from the malloc/free paths. So further optimizations can be done by disabling memory poisoning on free (a trivial single liner) and, more interesting, by removing memset(0) from the malloc path, especially for short IO.</p> <p>Andrew Perepechko (c17827@cray.com) uploaded a new patch: <a href="https://review.whamcloud.com/37795" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37795</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13309" title="performance optimizations for brw" class="issue-link" data-issue-key="LU-13309"><del>LU-13309</del></a> ofd: optimize the brw codepath<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8a065689c6fe38154d742f740355e58ab9f8f9d2</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37758/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37758/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13309" title="performance optimizations for brw" class="issue-link" data-issue-key="LU-13309"><del>LU-13309</del></a> osd-ldiskfs: remove per-page object_get/put in brw<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: dc9f28a541aec49b1787a25169f56a78a4924ee4</p> <p>Andrew Perepechko (andrew.perepechko@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/37915" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37915</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13309" title="performance optimizations for brw" class="issue-link" data-issue-key="LU-13309"><del>LU-13309</del></a> osd: use per-cpu counters for brw_stats<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 9f839039830b9aac5bf9038c2124a1e7a2450b0a</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37795/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37795/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13309" title="performance optimizations for brw" class="issue-link" data-issue-key="LU-13309"><del>LU-13309</del></a> ofd: optimize the brw codepath<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 3beeb90cfa47b4753083d09760a6bd5ecaf58d76</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37786/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37786/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13309" title="performance optimizations for brw" class="issue-link" data-issue-key="LU-13309"><del>LU-13309</del></a> osd-ldiskfs: speedup osd_bufs_get/put<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: e790df5fa38d4e8928dd28ba9f250fec4c830786</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/37915/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37915/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-13309" title="performance optimizations for brw" class="issue-link" data-issue-key="LU-13309"><del>LU-13309</del></a> osd: use per-cpu counters for brw_stats<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 787c1884e6451ae764568ade3658e537dcc19097</p> <p>Landed for 2.15</p> Related LU-13419 LU-15532 LU-13542 LU-12179 Development Rank 1|i00ujb: Rank (Obsolete) 9223372036854775807