https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-17182 Lustre <p>Using <tt>lctl pool_add FS OST0 OST1...</tt> is way slower than using an hostlist expression like <tt>OST<span class="error">&#91;0-7&#93;</span></tt>, see an example below:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> [root@localhost ~]# time lctl pool_add fs.pool_test fs-OST[0-7] OST fs-OST0000_UUID added to pool fs.pool_test OST fs-OST0001_UUID added to pool fs.pool_test OST fs-OST0002_UUID added to pool fs.pool_test OST fs-OST0003_UUID added to pool fs.pool_test OST fs-OST0004_UUID added to pool fs.pool_test OST fs-OST0005_UUID added to pool fs.pool_test OST fs-OST0006_UUID added to pool fs.pool_test OST fs-OST0007_UUID added to pool fs.pool_test real 0m9.008s user 0m0.000s sys 0m0.007s [root@localhost ~]# time lctl pool_add fs.pool_test OST0000 OST0001 OST0002 OST0003 OST0004 OST0005 OST0006 OST0007 OST fs-OST0000_UUID added to pool fs.pool_test OST fs-OST0001_UUID added to pool fs.pool_test OST fs-OST0002_UUID added to pool fs.pool_test OST fs-OST0003_UUID added to pool fs.pool_test OST fs-OST0004_UUID added to pool fs.pool_test OST fs-OST0005_UUID added to pool fs.pool_test OST fs-OST0006_UUID added to pool fs.pool_test OST fs-OST0007_UUID added to pool fs.pool_test real 1m7.024s user 0m0.004s sys 0m0.014s </pre> </div></div> <p>One could expect that both command to have the same runtime</p> LU-17182

lctl pool_add is slow when using individual OST

Bug Minor Resolved Fixed Feng Lei Raphael Druon Wed, 11 Oct 2023 13:45:29 +0000 Mon, 22 Jan 2024 16:06:46 +0000 Wed, 25 Oct 2023 20:02:46 +0000 Lustre 2.16.0 0 6 <p>Feng Lei</p> <p>Could you please investigate?</p> <p>Thanks</p> <p>Peter</p> <p>Yes, this is kind of a "known" problem - the "<tt>lctl pool&#42;</tt>" commands are all waiting on the update of the pool status to arrive at the client, which is asynchronous and takes several seconds to complete. You can see that with the individual OST list that it takes 67s, which is almost exactly 8x 9s taken to do all of them at once.</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=adilger" class="user-hover" rel="adilger">adilger</a> Possible to treat the second version (individual OST lists) the same as the first (indexset)? I.E. both versions run in parallel?</p> <p>I was going to say that the main problem is that the "add one at a time" case doesn't know whether there will be later commands run or not. They are all treated separately, and the command <b>explicitly</b> waits for the pool layout to be updated. That was added because the pool command would return without the actual update, and give the admin a false impression that the pool had been successfully updated, when it wasn't always the case.</p> <p>One option would be to add an "<tt>&#45;&#45;async</tt>" option to the pool commands that skips calling <tt>check_pool_cmd_result()</tt>, so that if you know a number of them will be executed (e.g. from EMF) that the preliminary ones are executed without waiting (and have a low chance of being done incorrectly by a user), and EMF can check the results afterward.</p> <p><b>However</b>, I now see that the second case is not "<tt>lctl pool_add ...; lctl pool_add ...; ...</tt>" but rather a single execution with multiple OSTs on the command-line. It should be possible to handle this with a single check at the end. It looks like <tt>jt_pool_cmd()</tt> would need to build up the OST list and execute all of the adds (removes) at once, instead of calling <tt>check_pool_cmd_result()</tt> for each argument separately.</p> <p>"Feng Lei &lt;flei@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/c/fs/lustre-release/+/52654" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/52654</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17182" title="lctl pool_add is slow when using individual OST" class="issue-link" data-issue-key="LU-17182"><del>LU-17182</del></a> utils: pool_add send OSTs in one batch<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: c6e77f54c450d2ca5283af05e54d171006e25a70</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/c/fs/lustre-release/+/52654/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/c/fs/lustre-release/+/52654/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-17182" title="lctl pool_add is slow when using individual OST" class="issue-link" data-issue-key="LU-17182"><del>LU-17182</del></a> utils: pool_add send OSTs in one batch<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: c8963c4935168c749896664e40aa4d11be90e0c3</p> <p>Landed for 2.16</p> Related Development Rank 1|i03y3j: Rank (Obsolete) 9223372036854775807 Severity