https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11546 Lustre <p>Now that e2fsprogs-1.44.3 has support for <tt>large_dir</tt>, testing and enabling the <tt>large_dir</tt> support on MDTs would allow single directories to exceed the ~10M limit currently imposed by the 2-level htree. This should be done automatically for new MDTs, and the existing error message in <tt>ext4_dx_add_entry()</tt> should be updated to directly reference the <tt>large_dir</tt> feature by name instead of just "Large directory", and explain that it should be enabled by tune2fs.</p> LU-11546

enable large_dir support for MDTs

Bug Minor Resolved Fixed Dongyang Li Andreas Dilger LTS12 Thu, 18 Oct 2018 22:43:27 +0000 Wed, 25 May 2022 05:46:44 +0000 Tue, 12 Nov 2019 06:37:17 +0000 Lustre 2.12.0 Lustre 2.13.0 Lustre 2.14.0 Lustre 2.12.8 0 7 <p>Note that I <b>don't</b> think that <tt>large_dir</tt> should be used for OSTs. For very large OSTs that exceed the 10M-entry limit for the <tt>O/0/d&#42;</tt> directories, I think it makes more sense to have the MDTs create fewer than <tt>LUSTRE_DATA_SEQ_MAX_WIDTH</tt> (= 4B) objects per OST sequence, and have the OSTs create new object directories <tt>O/&lt;seq&gt;/d&#42;</tt> for each sequence (which they already do for DNE when multiple MDTs are creating objects on the OST). This will allow the older object directory blocks to drop out of RAM as they become less used, and eventually those directories could be removed when they become empty.</p> <p>Having a single huge directory for objects means that the directory leaf blocks are updated totally randomly, and must always fit into RAM, or cause high read/write IOPS to the OST storage when there are lots of objects on a single OST. That is very undesirable, since it will typically be HDD-based OSTs that are so large they need more than 320M objects in a single filesystem (10M entries/directory * 32 directories).</p> <p>The "optimizing for more than 320M objects per OST per MDT" issue is being tracked in <a href="https://jira.whamcloud.com/browse/LU-11912" title="reduce number of OST objects created per MDS Sequence" class="issue-link" data-issue-key="LU-11912"><del>LU-11912</del></a>. This ticket is only for testing and enabling <tt>large_dir</tt> by default on filesystems.</p> <p>Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/35358" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35358</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11546" title="enable large_dir support for MDTs" class="issue-link" data-issue-key="LU-11546"><del>LU-11546</del></a> tests: enable large_dir support for tests<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 2d2209f33d9092e2de0a57f55c46950f5fee0fd5</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/35358/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35358/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11546" title="enable large_dir support for MDTs" class="issue-link" data-issue-key="LU-11546"><del>LU-11546</del></a> tests: enable large_dir support for tests<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 5a02d431f4a0a80915afa19c065df29c61e26ec9</p> <p>Dongyang, can you please make a patch to enable <tt>large_dir</tt> on MDTs when they are formatted by <tt>mkfs.lustre</tt>. This can go into master once 2.14 opens (in the next few weeks), and then likely backported to 2.12.4. </p> <p>Li Dongyang (dongyangli@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/36555" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36555</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11546" title="enable large_dir support for MDTs" class="issue-link" data-issue-key="LU-11546"><del>LU-11546</del></a> utils: enable large_dir for ldiskfs<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 4c703152e418707a2ec2be607c07329d80b19e80</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36555/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36555/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11546" title="enable large_dir support for MDTs" class="issue-link" data-issue-key="LU-11546"><del>LU-11546</del></a> utils: enable large_dir for ldiskfs<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: cd1faa0124f21e12a5ecd83c709c13918264fc86</p> <p>Landed for 2.14</p> <p>Minh Diep (mdiep@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36780" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36780</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11546" title="enable large_dir support for MDTs" class="issue-link" data-issue-key="LU-11546"><del>LU-11546</del></a> tests: enable large_dir support for tests<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 8059fd20860c57eaa224b9d30219000e3127d586</p> <p>Minh Diep (mdiep@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/36781" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36781</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11546" title="enable large_dir support for MDTs" class="issue-link" data-issue-key="LU-11546"><del>LU-11546</del></a> utils: enable large_dir for ldiskfs<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 823fc815fe3c5e3a7e2dd8ae8f8aa4d94301c35f</p> <p>It would be nice to have this patch landed into 2.12 at some point. We just used 2.12.5 RC1 to format a MDT and large_dir was not set. With DNE and especially if we use <tt>lfs migrate -m</tt>, <tt>large_dir</tt> becomes quickly mandatory on MDTs.</p> <p>Stephane, it is easy enough to set after formatting - "<tt>tune2fs -O large_dir &lt;dev&gt;</tt>". The holdup with landing the patch is that the tests written for this feature don't pass. That is mostly a problem with the tests themselves (they don't pass on master either), so either the change should be rebased to not depend on the tests, or the tests should be fixed.</p> <p>OK! We actually used <tt>mkfs.lustre -O large_dir,project</tt> and it worked fine. It's also easy to set using <tt>tune2fs</tt> like you said so not super critical to have it by default indeed. Thanks for the heads-up regarding the tests. Note that we have been using <tt>large_dir</tt> for months now on Fir's MDTs (2.12.x).</p> <p>"Oleg Drokin &lt;green@whamcloud.com&gt;" merged in patch <a href="https://review.whamcloud.com/36781/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36781/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11546" title="enable large_dir support for MDTs" class="issue-link" data-issue-key="LU-11546"><del>LU-11546</del></a> utils: enable large_dir for ldiskfs<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: 638638e3481f6dd3f6830e8e272362d8922c52f7</p> Related LU-11915 LU-1365 LU-11440 LU-11912 LU-6824 LU-12892 LU-14345 LU-14734 LU-12406 LU-10329 Development Rank 1|i004kf: Rank (Obsolete) 9223372036854775807 Severity