[LU-5501] new e2fsprogs causes llverfs build failures

• Type: Bug
• Resolution: Duplicate
• Priority: Major
• Affects Version/s: Lustre 2.5.3
• Environment: e2fsprogs-devel 1.42.11.wc2
• Severity: 3
• Assignee: Jian Yu
• Reporter: Bob Glossman (Inactive)

Description

Some lustre code uses #include files coming from the e2fsprogs rpms, in particular the #include files in /usr/include/ext2fs. With the new version #include files from the latest v1.42.11.wc2 e2fsprogs some strictly user level code that uses those #incudes fail. The only thing that stops this bug from being a Blocker is that it isn't seen when building lustre in master. I think that's due to some of the restructuring of lustre #include that have happened lately in master. It is seen when building in b2_5 and earlier branches.

Pretty sure this is due to changes in e2fsprogs #includes as it is only seen when the new 1.42.11 is installed, not seen with 1.42.9 and earlier. example errors:

gcc -DHAVE_CONFIG_H -I. -I../..  -D__arch_lib__ -D_LARGEFILE64_SOURCE=1 -DLUSTRE
_UTILS=1 -D_FILE_OFFSET_BITS=64 -include /home/bogl/other-lustre/config.h -I/hom
e/bogl/other-lustre/libcfs/include -I/home/bogl/other-lustre/lnet/include -I/hom
e/bogl/other-lustre/lustre/include  -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror 
-MT llverfs.o -MD -MP -MF .deps/llverfs.Tpo -c -o llverfs.o llverfs.c
In file included from /usr/include/e2p/e2p.h:14,
                 from llverfs.c:97:
/usr/include/ext2fs/ext2_fs.h:129: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:137: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:151: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:170: error: expected specifier-qualifier-list before ‘__u32’

For some reason that's not entirely clear to me generic typedefs like __u32 aren't getting pulled in correctly when building strictly userspace code, like in lustre/utils, when #includes from /usr/include/ext2fs are used.

Issue Links

• is related to: LU-5018 e2fsprogs build failed
• is related to: LU-4606 Lustre hard codes libzfs.so.1 in lustre/utils/mount_utils_zfs.c

Comments

[Bob Glossman (Inactive)]

Have verified e2fsprogs with 11901 patch builds OK and generates 1.42.12.wc1 version rpms. With that version installed both master and current b2_5 (without patch 11840) build OK, no build failures in lustre/utils seen. So far have only confirmed this on latest el6.5. Will gradually do other versions and distros as time permits.

[Andreas Dilger]

I've also pushed http://review.whamcloud.com/11901 to temporarily revert the e2fsprogs 5023510dd7 commit from the master-lustre branch. This will allow building e2fsprogs against Lustre 2.5.3 until change 11840 lands and is released.

The 11901 patch is based on the lustre patch series rebased onto the upstream e2fsprogs-1.42.12 release, and will build as 1.42.12.wc1.

[Bob Glossman (Inactive)]

I confirm that building b2_5 with http://review.whamcloud.com/11840 added works OK with new e2fsprogs #includes. I agree this is a better solution than backing out the e2fsprogs mod.

[Andreas Dilger]

It looks like this problem is isolated to b2_5's use of libtool to build the llverfs.c and llverdev.c programs. Testing on b2_4 does not show this problem, even though HAVE___U32 and HAVE_EXT2FS_EXT2FS_H are defined in config.h. That is because the lustre/utils/Makefile uses a "normal" build line that does not implicitly include config.h (which is really intended for the kernel) into userspace programs:

gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverfs llverfs.o
gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverdev llverdev.o  -lblkid

I did my e2fsprogs testing on my test system running master (~2.6.0) and on my production system running b2_4 (where the crazy libtool inclusion didn't exist), so I didn't see this problem. The dependency on libtool was added to master via LU-4606 change http://review.whamcloud.com/10193 (commit v2_5_59_0-19-g3e8c354) and on b2_5 via http://review.whamcloud.com/11297 (commit v2_5_2_0-32-g0d77156). The HAVE___U32 et.al. #defines were removed from configure on master via LU-5018 change http://review.whamcloud.com/10258 so the only affected version is actually 2.5.3 AFAIK.

It may be that the only work needed is to cherry-pick http://review.whamcloud.com/10258 onto b2_5 for 2.5.4, and in the meantime the patch affecting e2fsprogs can be temporarily reverted.

I've pushed a patch to b2_5 that cherry-picks 10258 (resolving a few conflicts): http://review.whamcloud.com/11840

[Bob Glossman (Inactive)]

output of compile of llverfs.c with -E

[Bob Glossman (Inactive)]

include file from e2fsprogs 1.42.11.wc2 present when b2_5 build fails

[Bob Glossman (Inactive)]

Andreas, when I build b2_5 I see the following configure status

checking ext2fs/ext2fs.h usability... no
checking ext2fs/ext2fs.h presence... yes
configure: WARNING: ext2fs/ext2fs.h: present but cannot be compiled
configure: WARNING: ext2fs/ext2fs.h:     check for missing prerequisite headers?
configure: WARNING: ext2fs/ext2fs.h: see the Autoconf documentation
configure: WARNING: ext2fs/ext2fs.h:     section "Present But Cannot Be Compiled"
configure: WARNING: ext2fs/ext2fs.h: proceeding with the preprocessor's result
configure: WARNING: ext2fs/ext2fs.h: in the future, the compiler will take precedence
configure: WARNING:     ## ----------------------------------------- ##
configure: WARNING:     ## Report this to http://bugs.whamcloud.com/ ##
configure: WARNING:     ## ----------------------------------------- ##
checking for ext2fs/ext2fs.h... yes

Note that in spite of the WARNINGS, similar to the ones you show, mine eventually says "checking for ext2fs/ext2fs.h... yes" anyway. My config.h therefore has:

#define HAVE_EXT2FS_EXT2FS_H 1

I will attach the /usr/include/ext2fs/ext2_types-x86_64.h present when the b2_5 build fails.

[Andreas Dilger]

Bob, what does the installed /usr/include/ext2fs/ext2_types-x86_64.h look like? When building b2_5 against e2fsprogs-1.42.11.wc2 I see in the configure output:

checking ext2fs/ext2fs.h usability... no
checking ext2fs/ext2fs.h presence... yes
configure: WARNING: ext2fs/ext2fs.h: present but cannot be compiled
configure: WARNING: ext2fs/ext2fs.h:     check for missing prerequisite headers?
configure: WARNING: ext2fs/ext2fs.h: see the Autoconf documentation
configure: WARNING: ext2fs/ext2fs.h:     section "Present But Cannot Be Compiled"
configure: WARNING: ext2fs/ext2fs.h: proceeding with the compiler's result
configure: WARNING:     ## ----------------------------------------- ##
configure: WARNING:     ## Report this to http://bugs.whamcloud.com/ ##
configure: WARNING:     ## ----------------------------------------- ##
checking for ext2fs/ext2fs.h... no

which is probably why HAVE_EXT2FS_EXT2FS_H is not defined on my test system where I develop e2fsprogs. My config.h does show HAVE___U32 is declared, so the ext2_types.h header should just skip these definitions entirely.

It might be useful to compile with "-E" to see what the actual code looks like after CPP is finished.

[Bob Glossman (Inactive)]

Andreas, strongly suspect the HAVE_EXT2FS_EXT2FS_H is in fact coming from the lustre config.h. Present in your example compile line as: "-include /usr/src/lustre-head/config.h"

That constant is #define'd in both my master build, where the build works, and the b2_5 build, where the build fails.

Can't say why the #includes aren't working as you describe, it seems like they should. I do confirm that backing out the 1 e2fsprogs commit that changes the ext2fs #includes does fix the problem.

[Andreas Dilger]

Could you please explain the actual problem further? The __u32 type is declared in ext2_types.h unless HAVE___U32 is already defined. In the older code, it unconditionally declared the __u32 type, even if it was already declared elsewhere.

This shouldn't affect the llverfs.c and llverdev.c programs, since they don't use __u32 themselves. I'm also wondering where the HAVE_EXT2FS_EXT2FS_H define is coming from? I see that the programs themselves don't include config.h and at least on my system this isn't defined by the build environment:

gcc -DHAVE_CONFIG_H -I. -I../..  -D__arch_lib__ -D_LARGEFILE64_SOURCE=1 -DLUSTRE_UTILS=1 -D_FILE_OFFSET_BITS=64 -include /usr/src/lustre-head/config.h -I/usr/src/lustre-head/libcfs/include -I/usr/src/lustre-head/lnet/include -I/usr/src/lustre-head/lustre/include  -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -MT llverdev.o -MD -MP -MF .deps/llverdev.Tpo -c -o llverdev.o llverdev.c
mv -f .deps/llverdev.Tpo .deps/llverdev.Po
/bin/sh ../../libtool --tag=CC   --mode=link gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverdev llverdev.o -lext2fs -lblkid -lkeyutils
libtool: link: gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -o llverdev llverdev.o  -L/usr/src/lustre-head/lnet/utils -lext2fs -lblkid -lkeyutils

Maybe this has been a longstanding problem, but it is only exposed on some new platform because HAVE_EXT2FS_EXT2FS_H is now defined by the build environment?

[Jian Yu]

Bob verified that installing e2fsprogs built with the above change backed out did make the llverfs and llverdev build problem go away. I'm investigating if there is a way to fix it instead of reverting it.