https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-15850 Lustre <p>The MDT QOS should always be used for subdirectories created in a parent that has round-robin activated, if the MDT space balance exceeds <tt>qos_threshold_rr</tt>. Otherwise, subdirectories in that directory tree will suddenly change from r-r to being "sticky" on a single MDT, which significantly changed the behavior and load distribution across MDTs. The "threshold by depth" should only be used for directories that would otherwise have always been created on the parent already. </p> <p>Related to this, it should be possible to tune the weighting of subdirectories by depth so that this can be adjusted without recompiling the code. </p> LU-15850

MDT QOS should always be used for round-robin directories.

Improvement Minor Resolved Fixed Lai Siyao Andreas Dilger MON Thu, 12 May 2022 18:00:28 +0000 Fri, 28 Oct 2022 17:44:33 +0000 Fri, 5 Aug 2022 20:20:17 +0000 Lustre 2.15.0 Lustre 2.16.0 0 7 <p>Lai, I was trying to test my patch to fix the "use space balance for RR directories" issue, but found something very wrong with the max-inherit and max-inherit-rr code, when used with <b>explicitly</b> inherited default layouts (i.e. layouts set on a non-root directory and copied down the tree while decrementing max-inherit-rr).</p> <p>When used with the <b>implicitly</b> inherited layout from the root directory, the max-inherit-rr value is copied from the ROOT directory, and is compared against <tt>lli_dir_depth</tt> (which increases with directory depth):</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> <span class="code-keyword">if</span> (lsm-&gt;lsm_md_max_inherit_rr != LMV_INHERIT_RR_NONE &amp;&amp; (lsm-&gt;lsm_md_max_inherit_rr == LMV_INHERIT_RR_UNLIMITED || lsm-&gt;lsm_md_max_inherit_rr &gt;= lli-&gt;lli_dir_depth)) op_data-&gt;op_flags |= MF_RR_MKDIR; </pre> </div></div> <p>This works as expected because <tt>lsm_md_max_inherit_rr</tt> is constant when <b>implicitly inherited</b> from the root directory, and <tt>lli_dir_depth</tt> is increasing by directory depth. <b>However</b>, if <tt>lsm_md_max_inherit_rr</tt> is on an <b>explicitly copied</b> default layout on a directory, then <tt>lsm_md_max_inherit_rr</tt> is decremented by one for each level <b>FROM THE FILESYSTEM ROOT</b>, while <tt>lli_dir_depth</tt> is incremented by one for each level. So in this second case, these values can be totally unrelated and the comparison is meaningless. For example, if the directory is 10 deep from the filesystem root, then <tt>lli_dir_depth</tt> must be &gt;= 10 <b>on the parent directory</b>.</p> <p>I'm thinking something like "store <tt>(lsm_md_max_inherit_rr + parent-&gt;lli_dir_depth)</tt> in memory on the parent directory, so that the child directory (with <tt>(child-&gt;lli_dir_depth = parent-&gt;lli_dir_depth + 1)</tt>, so that the "<tt>parent-&gt;lli_dir_depth</tt>" value cancels out and the above check works properly. However, it doesn't seem very obvious yet how that will be implemented properly.</p> <p>Indeed, dir-depth only considered ROOT. We may convert lsm_md_max-inherit and lsm_md_max-inherit-rr to absolute dir depth to ROOT in lsm unpack, then the comparison with lli_dir_depth will be opaque.</p> <p>Andreas, below code is for filesystem-wide default LMV only:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> if (lsm-&gt;lsm_md_max_inherit_rr != LMV_INHERIT_RR_NONE &amp;&amp; (lsm-&gt;lsm_md_max_inherit_rr == LMV_INHERIT_RR_UNLIMITED || lsm-&gt;lsm_md_max_inherit_rr &gt;= lli-&gt;lli_dir_depth)) op_data-&gt;op_flags |= MF_RR_MKDIR; </pre> </div></div> <p>This looks not to be an issue.</p> <p>I was trying to check <tt>MF_RR_MKDIR</tt> to see if the directory has round-robin allocation enabled, so that the "stay on parent" check in <tt>lmv_locate_tgt_qos()</tt> would not be used if the parent is r-r. </p> <p>"Lai Siyao &lt;lai.siyao@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/47576" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47576</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15850" title="MDT QOS should always be used for round-robin directories. " class="issue-link" data-issue-key="LU-15850"><del>LU-15850</del></a> mdt: pack default LMV in open reply<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: c31475758946fd5f32382b5415ab7c0c1b46913c</p> <p>"Lai Siyao &lt;lai.siyao@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/47577" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47577</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15850" title="MDT QOS should always be used for round-robin directories. " class="issue-link" data-issue-key="LU-15850"><del>LU-15850</del></a> llite: pass dmv inherit depth instead of dir depth<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: d33bd101c7b2a9184615d6ff4751fe8d6222283b</p> <p>"Lai Siyao &lt;lai.siyao@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/47578" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47578</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15850" title="MDT QOS should always be used for round-robin directories. " class="issue-link" data-issue-key="LU-15850"><del>LU-15850</del></a> lmv: always space-balance r-r directories<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 9017448833a249ce29ea0c5d26c60e3d5dc03201</p> <p>"Lai Siyao &lt;lai.siyao@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/47679" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47679</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15850" title="MDT QOS should always be used for round-robin directories. " class="issue-link" data-issue-key="LU-15850"><del>LU-15850</del></a> llite: interop test with 2.14<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 03fba460bc0f1b6a747b792715ac7f8f059eacfe</p> <p>"Lai Siyao &lt;lai.siyao@whamcloud.com&gt;" uploaded a new patch: <a href="https://review.whamcloud.com/47789" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47789</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15850" title="MDT QOS should always be used for round-robin directories. " class="issue-link" data-issue-key="LU-15850"><del>LU-15850</del></a> llite: implicit default LMV inherit<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: cc65c9deaab82dfc7ab81235ba8c0339fe8fc73b</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/47576/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47576/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15850" title="MDT QOS should always be used for round-robin directories. " class="issue-link" data-issue-key="LU-15850"><del>LU-15850</del></a> mdt: pack default LMV in open reply<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: f6e4272fb0be5b798b7685bb40067e3f6877c8a5</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/47577/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47577/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15850" title="MDT QOS should always be used for round-robin directories. " class="issue-link" data-issue-key="LU-15850"><del>LU-15850</del></a> llite: pass dmv inherit depth instead of dir depth<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: c23c68a52a04369101db2bd3b1d3da23025fcf48</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/47578/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/47578/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-15850" title="MDT QOS should always be used for round-robin directories. " class="issue-link" data-issue-key="LU-15850"><del>LU-15850</del></a> lmv: always space-balance r-r directories<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 37c1ddc34d3a1e61c5533f48cb29fe2258ca2907</p> <p>Landed for 2.16</p> Related LU-15910 LU-13440 LU-13439 LU-14762 LU-13417 Development Rank 1|i02ptj: Rank (Obsolete) 9223372036854775807