https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-17197 Lustre <p>A performance regression:</p> <p>Introduced MDTest Unique Directory Directory (-25%)/File Removal(-11%)/File Reads Regressions</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>mdtest -v -i 5 -p 120 -C -w 32768 -E -e 32768 -T -r -n 3120 -u -d &lt;dir&gt; ... mdtest -v -i 5 -p 120 -C -E -T -r -n 3120 -u -d &lt;dir&gt; </pre> </div></div> <p>Also an IOR 4k sequential read (-6%)</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>ior -F -r -t 4k -D 180 -u -b 8g -o &lt;path&gt; ... Options: api : POSIX apiVersion : test filename : /mnt/kjlmo2/pkoutoupis/flash/test.05 access : file-per-process type : independent segments : 1 ordering in a file : sequential ordering inter file : no tasks offsets nodes : 21 tasks : 336 clients per node : 16 repetitions : 1 xfersize : 4096 bytes blocksize : 8 GiB aggregate filesize : 2.62 TiB stonewallingTime : 180 stoneWallingWearOut : 0 </pre> </div></div> LU-17197

Performance regression with "LU-15947 obdclass: improve precision of wakeups for mod_rpcs"

Bug Major Resolved Fixed Shaun Tancheff Shaun Tancheff Sat, 14 Oct 2023 09:01:18 +0000 Fri, 22 Dec 2023 18:53:37 +0000 Fri, 22 Dec 2023 18:53:37 +0000 Lustre 2.16.0 0 7 <p>"Shaun Tancheff &lt;shaun.tancheff@hpe.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/52698" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/52698</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17197" title="Performance regression with &quot;LU-15947 obdclass: improve precision of wakeups for mod_rpcs&quot;" class="issue-link" data-issue-key="LU-17197"><del>LU-17197</del></a> obdclass: revert improve precision of wakeups<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 8f1dd012fd651a05824919fff8591c3ae73d9cf3</p> <p>"Shaun Tancheff &lt;shaun.tancheff@hpe.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/52699" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/52699</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17197" title="Performance regression with &quot;LU-15947 obdclass: improve precision of wakeups for mod_rpcs&quot;" class="issue-link" data-issue-key="LU-17197"><del>LU-17197</del></a> ptlrpc: Sort waiters on close_req completion<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 3054382915f534e8fb4ed00e256b66fda2aab5a9</p> <p>25% is a really big regression.  I think the most likely cause is requests being handled in a different order.</p> <p>The patch you are reverting claimed to improve fairness but I now see that it does exactly the reverse.</p> <p>Previously most waiters were queued with wait_event_idle_exclusive which adds them to the end of the queue.  With my patch __add_wait_queue() is used, which added to the front.  That is bad - but easily fixed.</p> <p>Instead of this patch, could you please try changing the __add_wait_queue() in obd_get_mod_rpc_slot() to __add_wait_queue_entry_tail().  That should restore fairness and preserve all the other good properties of my patch.</p> <p>Thanks.</p> <p>"Neil Brown &lt;neilb@suse.de&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/52738" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/52738</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17197" title="Performance regression with &quot;LU-15947 obdclass: improve precision of wakeups for mod_rpcs&quot;" class="issue-link" data-issue-key="LU-17197"><del>LU-17197</del></a> obdclass: preserve fairness when waiting for rpc slot<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: d7f66cb67c025519ec743ac3594b9634562e2223</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/52738/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/52738/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17197" title="Performance regression with &quot;LU-15947 obdclass: improve precision of wakeups for mod_rpcs&quot;" class="issue-link" data-issue-key="LU-17197"><del>LU-17197</del></a> obdclass: preserve fairness when waiting for rpc slot<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b5fde4d6c02324a8511afe30d02eb2cf46ea799d</p> <p>"Etienne AUJAMES &lt;eaujames@ddn.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/53232" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/53232</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17197" title="Performance regression with &quot;LU-15947 obdclass: improve precision of wakeups for mod_rpcs&quot;" class="issue-link" data-issue-key="LU-17197"><del>LU-17197</del></a> obdclass: preserve fairness when waiting for rpc slot<br/> Project: fs/lustre-release<br/> Branch: b2_15<br/> Current Patch Set: 1<br/> Commit: 34de852e55233bc13d1e795a0a1912f5ac80e15b</p> <p>Fix merged for 2.16</p> Related LU-15947 Development Rank 1|i03ydb: Rank (Obsolete) 9223372036854775807 Severity