https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-14459 Lustre <p>Directory auto-split should be done when the <tt>dir_split_count</tt> is hit during file creation to minimize the number of entries that need to be moved, rather than afterward. Otherwise, the entries are being created on a single MDT first, and only being migrated on a later directory access. This is sub-optimal for two reasons:</p> <ul class="alternate" type="square"> <li>the initial file creation will be limited to a single MDT and not distribute the load across multiple MDTs, so just entry migration will still leave all the inodes on the original MDT</li> <li>the post-creation migration will need to move many more files/entries to the other MDTs than if the files are created on the correct MDT to start with</li> </ul> LU-14459

DNE3: directory auto split during create

Improvement Major Open Unresolved Lai Siyao Andreas Dilger dne3 Fri, 19 Feb 2021 23:01:29 +0000 Thu, 25 Aug 2022 01:18:31 +0000 0 7 <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/41713" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/41713</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> mdt: autosplit directory during create<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: bcdd4c972b8e8b3a2f53158740a7ff3999c895f0</p> <p>Lai, I started looking into this to see if it was easily implemented. I pushed the above patch with some preliminary changes, but it needs more work to be done properly.</p> <p>It looks like <tt>la_dirent_count</tt> is only fetched once at getattr time, but then never updated when directories are added/removed. That needs to be updated during file creation/removal so that the count is at least somewhat accurate as the directory changes. If it simplifies implementation, then <tt>la_dirent_count</tt> doesn't need to be perfect (e.g. it could be checked only when the name hash % 64 == 0, or whatever), since a growing directory should eventually increase it enough to cause a split, even if the split is a bit late, and for large directories it will eventually hit the stripe count limit and then an accurate count again doesn't matter.</p> <p>I wasn't sure exactly where to put the check for <tt>mdt_should_auto_split()</tt> and <tt>mdt_auto_split_add()</tt>. First I was thinking in <tt>mdt_create()</tt>, but that doesn't have <tt>la_dirent_count</tt> for <tt>parent</tt>, and adding a call to fetch it on each create seemed like a lot of added overhead. It might make sense to move <tt>la_dirent_count</tt> into <tt>struct mdt_object</tt> rather than <tt>struct lu_attrs</tt>, so that it is kept in memory with the directory instead of having to be fetched from the OSD repeatedly. The <tt>la_dirent_count</tt> could be fetched once for the <tt>mdt_object</tt> when it is first loaded, then updated in memory for each create/unlink in the directory (it isn't clear if splitting the directory is useful for hard links or not?).</p> <p>Minor note - I see <tt>mot_auto_split_disabled</tt> in <tt>struct mdt_object</tt>, but it doesn't look like this is used anywhere and could be removed? Or is there intended to be a way to force a directory to <b>not</b> be split (e.g. attribute set on the directory or layout)?</p> <p>The dirent count is maintained in <tt>osd_object.oo_dirent_count</tt>, which will be updated in creation/removal.</p> <p>The reason why it's not cached in <tt>mdt_object</tt> is that if the newly created sub file is remote object, its parent is remote, thus the cached dirent count can't be updated.</p> <p><tt>mot_auto_split_disabled</tt> is from legacy code, it's intended to stop retrying splitting a directory (maybe we should say both splitting and sub files migration) if some error occurred in auto-split, and it failed after several retries. But it's not implemented, and currently it will retry finish splitting directory once server notices an unfinished split.</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/42064" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/42064</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> mdt: trigger dir split in create/open<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 0edbcc3ce6de3f2c625c5f3be6805565a81fab51</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/43289" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43289</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> llite: init stripe pfid after dir split<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 768bc1b9e93ebc5c3c53612287abd3ca136c622c</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/43290" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43290</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> mdt: restripe parent may be a stripe<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 93711b9e00d28d235c2b10409bf135eb6c6e5944</p> <p>Lai Siyao (lai.siyao@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/43291" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43291</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> mdt: support fixed directory layout<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 0c0995c016c49f21b1dea938cdb71c6ccb7bf658</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/43684" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43684</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> lmv: change default hash type to crush<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: e859f57768b8b7ee2e44924c2b36e39a7f0f323a</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/43684/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43684/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> lmv: change default hash type to crush<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: bb60caa1c6e7c14c201916dc0423442d10c86a27</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/43289/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43289/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> llite: reset pfid after dir migration<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: abbe545a63b304e803ee62443dd65f1feeed15cd</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/43290/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43290/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> mdt: restripe parent may be a stripe<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: a84efc8607ae8057499a8800699f336e821b03d8</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/43291/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43291/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-14459" title="DNE3: directory auto split during create" class="issue-link" data-issue-key="LU-14459">LU-14459</a> mdt: support fixed directory layout<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 4c2514f4832801374092f3a48c755248af345566</p> <p>The performance impact of the auto-split during mdtest is too high, since this moves all of the entries while the benchmark is running, and DNE distributed transactions cause a lot of sync operations. It would be better to delay the entry move, like put migrations for directory split onto a queue, and only move those entries when the RPC threads are idle (maybe with a "low priority" RPC queue that is only handled when high/normal RPCs are finished?).</p> <p>That would avoid blocking the benchmark RPCs, but still split the directory early so that the metadata IOPS of the directory increase during the test, and "most" entries are created on the right MDT, with fewer entries need to be moved later on.</p> Cloners Duplicate LU-14467 Related LU-14146 LU-11025 LU-14464 LU-14466 LU-15692 LU-15720 LU-15502 Development Rank 1|i01n6n: Rank (Obsolete) 9223372036854775807