[LU-1365] Implement ldiskfs LARGEDIR support for e2fsprogs

Duplicate — Fri, 4 Jun 2021 21:25:48 +0000

This INCOMPAT_LARGEDIR feature allows larger directories to be created in ldiskfs, both with directory sizes over 2GB and and a maximum htree depth of 3 instead of the current limit of 2. These features are needed in order to exceed the current limit of approximately 10M entries in a single directory. The INCOMPAT_LARGEDIR feature was added to ldiskfs as part of the pdirops ~~LU-50~~ coding, but was not part of that project. As there is currently no mke2fs, e2fsck, or tune2fs support for INCOMPAT_LARGEDIR, this feature is disabled by default when creating a new ldiskfs filesystem as it would otherwise make the filesystem unrecoverable in the case that e2fsck needs to be run on it.

Tasks that need to be completed before INCOMPAT_LARGEDIR can be used:

add support for the INCOMPAT_LARGEDIR and "large_dir" features to mke2fs/tune2fs
add conf-sanity.sh test LARGEDIR and 3-level htree for local ldiskfs with 1kB blocksize up to 100k entries with 255-byte names (3-level exceeded at 48k entries). This might be done using a smaller number of hard-linked inodes (nlink_max = 65000), to avoid overhead of accessing and caching a large number of different inodes.
add parallel-scale.sh test LARGEDIR and >2GB directories with Lustre using 255-byte names and 10M entries (2GB exceeded at 4M entries, 4GB exceeded at 8M entries). This might be done using a smaller number of hard-linked inodes (nlink_max = 65000), to avoid overhead of accessing and caching a large number of different inodes.
e2fsck support for 3-level htree
e2fsck support for directories larger than 2GB (using i_size_hi consistently for S_IFDIR() inodes)
e2fsprogs regression test test for 3-level/2GB+ htree e2fsck, corruptions
e2fsprogs add LARGEDIR feature to "tests/f_random_corruption"
port the ext4-large-dir.patch with the INCOMPAT_LARGEDIR features (>2GB, 3-level htree) to the upstream kernel and submit to linux-ext4 list for review
submit e2fsprogs patches to linux-ext4 list for review
solicit testing of feature from community
some time after e2fsprogs is released and available for download, a patch is needed to enable large_dir on new filesystems with mkfs.lustre
updates to the user manual and release notes to describe how to enable the large_dir feature with tune2fs

Whamcloud Community JIRA

[LU-1365] Implement ldiskfs LARGEDIR support for e2fsprogs