[LU-11246] New lustre e2fsprogs 1.44 issues

• Type: Bug
• Resolution: Duplicate
• Priority: Critical
• Environment: Building and use latest modifies e2fsprogs for lustre.
• Severity: 3
• Assignee: Dongyang Li
• Reporter: James A Simmons

Description

With the need to build e2fsprogs for ARM and Power8 I also attempted to build it for my general RHEL7 systems. I discovered a few things. First I think we can eliminate the extra spec files. If you install lsb_release for your distro it will build with the default spec file. A yum install redhat-release will do it. On my build machine I have libfuse-devel installed which broke the e2fsprogs. Even a ./configure --disable-fuse2fs didn't help. Lastly for my new target testbed the disk are 75TB in size so when I attempted to build a file system I saw the following error: 

mke2fs 1.44.3.wc1 (23-July-2018)

mke2fs: Size of device (0x448000000 blocks) /dev/mapper/crius-ddn-l12 too big to be expressed in 32 bits using a blocksize of 4096.

It looks like a -o 64 option will be needed.

Issue Links

• is blocked by: LU-11268 mdc_intent_getxattr_pack() allocates too large name and value length buffers in interop case
• is related to: LU-11215 conf-sanity test_61: Invalid filesystem option set: large_xattr, Clearing filesystem feature 'ea_inode' not supported
• is related to: LU-10997 Ubuntu 18 support
• is related to: LU-11440 Make e2fsprogs-1.44.3-wc1 release
• is related to: LU-1732 enable wide striping by default
• is related to: LU-6387 Add Power8 support to Lustre
• is related to: LU-11200 Centos 8 arm64 server support

Comments

[Dongyang Li]

400 is EXT4_FEATURE_INCOMPAT_EA_INODE, which should be there in ldiskfs

what does the mkfs.lustre output say? like the options it passed to mke2fs?

also what does dumpe2fs -h say?

[James A Simmons]

I managed to get my Power8 nodes running the lustre server code but using the latest e2fsprogs I'm seeing the following:

[ 1049.122763] Lustre: DEBUG MARKER: mkfs.lustre --mgsnode=10.37.202.6@o2ib1 --fsname=lustre --mdt --index=0 --param=sys.timeout=20 --param=mdt.identity_upcall=/usr/sbin/l_getidentity --backfstype=ldiskfs --device-size=250000 --reformat /dev/nvme0n1

[ 1049.486023] LDISKFS-fs (nvme0n1): Couldn't mount because of unsupported optional features (400)

Was something missed?

[Dongyang Li]

I was overwriting the commits, we want to keep the number of patches on top of e2fsprogs upstream to the minimum.

One way to check is to look into the spec.in file, see if you have --disable-fuse2fs in the configure section.

[James A Simmons]

Last commit I see is "e2fsprogs: fix compile error and warnings for old gcc versions". Which commit is it?

[Dongyang Li]

James, it's the same master-lustre-test branch, do a git pull or clone you should be able to see it.

[James A Simmons]

Which repo did you push the fuse to?  I'm working off the master-lustre-test branch. 

@Andreas: I'm building the file system with mkfs.lustre using the tip of pre-2.12

[Dongyang Li]

We do need the specs for the platforms to overcome the differences,

e.g. several test case will always fail on a default SLES11 buildbox because it picks ext3 as the root filesystem. we need to apply extra patches and skip some tests for SLES only.

When I first made the specs I put fuse-devel as a build dependency, but 

1. fuse2fs is not useful to us, it can not mount fs with ea_inode

2. the build boxes from jenkins doesn't have fuse-devel by default

so I've removed them from the specs, however the --disable-fuse2fs should go into the configure options in the specs. I've just fixed that in the git repo.

[Andreas Dilger]

With the need to build e2fsprogs for ARM and Power8 I also attempted to build it for my general RHEL7 systems. I discovered a few things. First I think we can eliminate the extra spec files. If you install lsb_release for your distro it will build with the default spec file. A yum install redhat-release will do it.

Firstly, I suspect that the yum install will not work for SLES, and it isn't clear if it would work for RHEL6? How does it know to get the right .spec file for the distro? Also, I suspect the vendor .spec file will not have the correct contact URLs and such, so it isn't clear to me how that would work. I'm not against reducing our maintenance overhead by eliminating patches, but you'd need to explain a bit more about what this is doing before I can understand whether it is actually something we want to change.

On my build machine I have libfuse-devel installed which broke the e2fsprogs. Even a ./configure --disable-fuse2fs didn't help.

lidongyang had a patch to add a configure check for libfuse-devel to skip this if it wasn't available. Maybe that is missing from the version that you were using? Or is the problem that this configure check is missing something and it still fails to build when libfuse-devel is not installed?

Lastly for my new target testbed the disk are 75TB in size so when I attempted to build a file system I saw the following error:

mke2fs 1.44.3.wc1 (23-July-2018)

mke2fs: Size of device (0x448000000 blocks) /dev/mapper/crius-ddn-l12 too big to be expressed in 32 bits using a blocksize of 4096.

It looks like a -o 64 option will be needed.

I think we already add "-O 64bit" from mkfs.lustre by default for OSTs. Are you running mke2fs directly on the devices?

static int enable_default_ext4_features()
{
        /* Enable large block addresses if the LUN is over 2^32 blocks. */
        if ((mop->mo_device_kb / (L_BLOCK_SIZE >> 10) > UINT32_MAX) &&
             is_e2fsprogs_feature_supp("-O 64bit") == 0)
                enable_64bit = 1;

This has been the default even for MDTs since commit v2_10_58_0-123-geb65c3a patch https://review.whamcloud.com/31037 "LU-10520 mkfs: enable extents for big MDT" because even though large MDTs were uncommon in the past, with DoM it is more likely that we will need large MDT filesystems. With a 75TB MDT holding 1B inodes, that is only about 64KB/file once the other filesystem overhead is taken into account.