[LU-11546] enable large_dir support for MDTs

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0, Lustre 2.12.8
• Affects Version/s: Lustre 2.12.0, Lustre 2.13.0
• Labels: LTS12
• Severity: 3
• Assignee: Dongyang Li
• Reporter: Andreas Dilger

Description

Now that e2fsprogs-1.44.3 has support for large_dir, testing and enabling the large_dir support on MDTs would allow single directories to exceed the ~10M limit currently imposed by the 2-level htree. This should be done automatically for new MDTs, and the existing error message in ext4_dx_add_entry() should be updated to directly reference the large_dir feature by name instead of just "Large directory", and explain that it should be enabled by tune2fs.

Issue Links

• is related to: LU-12892 Large directory feature is not enabled (REMOTE_PARENT_DIR)
• is related to: LU-14345 e2fsck of very large directories is broken
• is related to: LU-14734 enable large_dir on existing MDTs
• is related to: LU-12406 conf-sanity test 111 fails with ‘add mds1 failed with new params’
• is related to: LU-10329 DNE3: REMOTE_PARENT_DIR scalability
• is related to: LU-11915 conf-sanity test 115 is skipped or hangs
• is related to: LU-1365 Implement ldiskfs LARGEDIR support for e2fsprogs
• is related to: LU-11440 Make e2fsprogs-1.44.3-wc1 release
• is related to: LU-11912 reduce number of OST objects created per MDS Sequence
• is related to: LU-6824 improve error messages for dir htree limit

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/36781/
Subject: LU-11546 utils: enable large_dir for ldiskfs
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: 638638e3481f6dd3f6830e8e272362d8922c52f7

[Stephane Thiell]

OK! We actually used mkfs.lustre -O large_dir,project and it worked fine. It's also easy to set using tune2fs like you said so not super critical to have it by default indeed. Thanks for the heads-up regarding the tests. Note that we have been using large_dir for months now on Fir's MDTs (2.12.x).

[Andreas Dilger]

Stephane, it is easy enough to set after formatting - "tune2fs -O large_dir <dev>". The holdup with landing the patch is that the tests written for this feature don't pass. That is mostly a problem with the tests themselves (they don't pass on master either), so either the change should be rebased to not depend on the tests, or the tests should be fixed.

[Stephane Thiell]

It would be nice to have this patch landed into 2.12 at some point. We just used 2.12.5 RC1 to format a MDT and large_dir was not set. With DNE and especially if we use lfs migrate -m, large_dir becomes quickly mandatory on MDTs.

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36781
Subject: LU-11546 utils: enable large_dir for ldiskfs
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 823fc815fe3c5e3a7e2dd8ae8f8aa4d94301c35f

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36780
Subject: LU-11546 tests: enable large_dir support for tests
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 8059fd20860c57eaa224b9d30219000e3127d586

[Peter Jones]

Landed for 2.14

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36555/
Subject: LU-11546 utils: enable large_dir for ldiskfs
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cd1faa0124f21e12a5ecd83c709c13918264fc86

[Gerrit Updater]

Li Dongyang ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36555
Subject: LU-11546 utils: enable large_dir for ldiskfs
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4c703152e418707a2ec2be607c07329d80b19e80

[Andreas Dilger]

Dongyang, can you please make a patch to enable large_dir on MDTs when they are formatted by mkfs.lustre. This can go into master once 2.14 opens (in the next few weeks), and then likely backported to 2.12.4.