[LU-5501] new e2fsprogs causes llverfs build failures

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.5.3
• Type: Bug
• Priority: Major
• Reporter: Bob Glossman (Inactive)
• Assignee: Jian Yu
• Resolution: Duplicate

Environment

e2fsprogs-devel 1.42.11.wc2

• Severity: 3

Description

Some lustre code uses #include files coming from the e2fsprogs rpms, in particular the #include files in /usr/include/ext2fs. With the new version #include files from the latest v1.42.11.wc2 e2fsprogs some strictly user level code that uses those #incudes fail. The only thing that stops this bug from being a Blocker is that it isn't seen when building lustre in master. I think that's due to some of the restructuring of lustre #include that have happened lately in master. It is seen when building in b2_5 and earlier branches.

Pretty sure this is due to changes in e2fsprogs #includes as it is only seen when the new 1.42.11 is installed, not seen with 1.42.9 and earlier. example errors:

gcc -DHAVE_CONFIG_H -I. -I../..  -D__arch_lib__ -D_LARGEFILE64_SOURCE=1 -DLUSTRE
_UTILS=1 -D_FILE_OFFSET_BITS=64 -include /home/bogl/other-lustre/config.h -I/hom
e/bogl/other-lustre/libcfs/include -I/home/bogl/other-lustre/lnet/include -I/hom
e/bogl/other-lustre/lustre/include  -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror 
-MT llverfs.o -MD -MP -MF .deps/llverfs.Tpo -c -o llverfs.o llverfs.c
In file included from /usr/include/e2p/e2p.h:14,
                 from llverfs.c:97:
/usr/include/ext2fs/ext2_fs.h:129: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:137: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:151: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:170: error: expected specifier-qualifier-list before ‘__u32’

For some reason that's not entirely clear to me generic typedefs like __u32 aren't getting pulled in correctly when building strictly userspace code, like in lustre/utils, when #includes from /usr/include/ext2fs are used.

Issue Links

• is related to: LU-5018 e2fsprogs build failed
• is related to: LU-4606 Lustre hard codes libzfs.so.1 in lust...

Comments

[Andreas Dilger]

Bob, is it enough to just switch the order of these include files, so #include <ext2fs/ext2fs.h> comes before #include <e2p/e2p.h>? Could you please give this a try in your test environment and submit a patch in that case?

[Bob Glossman (Inactive)]

no, just changing the order of the #includes and putting <ext2fs/ext2fs.h> first doesn't help. build still fails.

[Peter Jones]

Yu, Jian is looking into this one

[Jian Yu]

'__u32' is defined in ext2fs/ext2_types.h, which is included in ext2fs/ext2_fs.h. I'll look at what changes were made on ext2fs/ext2_types.h.

[Jian Yu]

Hi Andreas,

I saw the following change on lib/ext2fs/ext2_types.h.in, which was committed on Jul 4, 2014:

commit 5023510dd7f0ab40a216d3600ab665e2053cc070
Author: Andreas Dilger <[e-mail]>
Date:   Fri Jul 4 23:34:10 2014 -0400

    blkid,ext2fs: avoid name clash with __u{8,16,32,64}
    
    Try to avoid name clashes with definitions of __u8, __u16, __u32,
    and __u64 in userspace, in case other headers also define these
    types.  Define HAVE___{S,U}{8,16,32,64} preprocessor macros to
    show that these types are already defined.
    
    This would avoid the need to check for _BLKID_TYPES_H in ext2_types.h
    and _EXT2_TYPES_H in blkid_types.h, but since older versions of these
    headers did not use HAVE___U8 et.al. keep these checks around for now.
    
    Report an error if there are no 64-bit types available.  The code
    will not compile if these are not available.
    
    Signed-off-by: Andreas Dilger <[e-mail]>
    Signed-off-by: Theodore Ts'o <[e-mail]>

It looks like this is the culprit that causes the issue in this ticket.

[Jian Yu]

Bob verified that installing e2fsprogs built with the above change backed out did make the llverfs and llverdev build problem go away. I'm investigating if there is a way to fix it instead of reverting it.

[Andreas Dilger]

Could you please explain the actual problem further? The __u32 type is declared in ext2_types.h unless HAVE___U32 is already defined. In the older code, it unconditionally declared the __u32 type, even if it was already declared elsewhere.

This shouldn't affect the llverfs.c and llverdev.c programs, since they don't use __u32 themselves. I'm also wondering where the HAVE_EXT2FS_EXT2FS_H define is coming from? I see that the programs themselves don't include config.h and at least on my system this isn't defined by the build environment:

gcc -DHAVE_CONFIG_H -I. -I../..  -D__arch_lib__ -D_LARGEFILE64_SOURCE=1 -DLUSTRE_UTILS=1 -D_FILE_OFFSET_BITS=64 -include /usr/src/lustre-head/config.h -I/usr/src/lustre-head/libcfs/include -I/usr/src/lustre-head/lnet/include -I/usr/src/lustre-head/lustre/include  -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -MT llverdev.o -MD -MP -MF .deps/llverdev.Tpo -c -o llverdev.o llverdev.c
mv -f .deps/llverdev.Tpo .deps/llverdev.Po
/bin/sh ../../libtool --tag=CC   --mode=link gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverdev llverdev.o -lext2fs -lblkid -lkeyutils
libtool: link: gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -o llverdev llverdev.o  -L/usr/src/lustre-head/lnet/utils -lext2fs -lblkid -lkeyutils

Maybe this has been a longstanding problem, but it is only exposed on some new platform because HAVE_EXT2FS_EXT2FS_H is now defined by the build environment?

[Bob Glossman (Inactive)]

Andreas, strongly suspect the HAVE_EXT2FS_EXT2FS_H is in fact coming from the lustre config.h. Present in your example compile line as: "-include /usr/src/lustre-head/config.h"

That constant is #define'd in both my master build, where the build works, and the b2_5 build, where the build fails.

Can't say why the #includes aren't working as you describe, it seems like they should. I do confirm that backing out the 1 e2fsprogs commit that changes the ext2fs #includes does fix the problem.

[Andreas Dilger]

Bob, what does the installed /usr/include/ext2fs/ext2_types-x86_64.h look like? When building b2_5 against e2fsprogs-1.42.11.wc2 I see in the configure output:

checking ext2fs/ext2fs.h usability... no
checking ext2fs/ext2fs.h presence... yes
configure: WARNING: ext2fs/ext2fs.h: present but cannot be compiled
configure: WARNING: ext2fs/ext2fs.h:     check for missing prerequisite headers?
configure: WARNING: ext2fs/ext2fs.h: see the Autoconf documentation
configure: WARNING: ext2fs/ext2fs.h:     section "Present But Cannot Be Compiled"
configure: WARNING: ext2fs/ext2fs.h: proceeding with the compiler's result
configure: WARNING:     ## ----------------------------------------- ##
configure: WARNING:     ## Report this to http://bugs.whamcloud.com/ ##
configure: WARNING:     ## ----------------------------------------- ##
checking for ext2fs/ext2fs.h... no

which is probably why HAVE_EXT2FS_EXT2FS_H is not defined on my test system where I develop e2fsprogs. My config.h does show HAVE___U32 is declared, so the ext2_types.h header should just skip these definitions entirely.

It might be useful to compile with "-E" to see what the actual code looks like after CPP is finished.

[Bob Glossman (Inactive)]

Andreas, when I build b2_5 I see the following configure status

checking ext2fs/ext2fs.h usability... no
checking ext2fs/ext2fs.h presence... yes
configure: WARNING: ext2fs/ext2fs.h: present but cannot be compiled
configure: WARNING: ext2fs/ext2fs.h:     check for missing prerequisite headers?
configure: WARNING: ext2fs/ext2fs.h: see the Autoconf documentation
configure: WARNING: ext2fs/ext2fs.h:     section "Present But Cannot Be Compiled"
configure: WARNING: ext2fs/ext2fs.h: proceeding with the preprocessor's result
configure: WARNING: ext2fs/ext2fs.h: in the future, the compiler will take precedence
configure: WARNING:     ## ----------------------------------------- ##
configure: WARNING:     ## Report this to http://bugs.whamcloud.com/ ##
configure: WARNING:     ## ----------------------------------------- ##
checking for ext2fs/ext2fs.h... yes

Note that in spite of the WARNINGS, similar to the ones you show, mine eventually says "checking for ext2fs/ext2fs.h... yes" anyway. My config.h therefore has:

#define HAVE_EXT2FS_EXT2FS_H 1

I will attach the /usr/include/ext2fs/ext2_types-x86_64.h present when the b2_5 build fails.

[Bob Glossman (Inactive)]

include file from e2fsprogs 1.42.11.wc2 present when b2_5 build fails

[Bob Glossman (Inactive)]

output of compile of llverfs.c with -E

[Andreas Dilger]

It looks like this problem is isolated to b2_5's use of libtool to build the llverfs.c and llverdev.c programs. Testing on b2_4 does not show this problem, even though HAVE___U32 and HAVE_EXT2FS_EXT2FS_H are defined in config.h. That is because the lustre/utils/Makefile uses a "normal" build line that does not implicitly include config.h (which is really intended for the kernel) into userspace programs:

gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverfs llverfs.o
gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverdev llverdev.o  -lblkid

I did my e2fsprogs testing on my test system running master (~2.6.0) and on my production system running b2_4 (where the crazy libtool inclusion didn't exist), so I didn't see this problem. The dependency on libtool was added to master via LU-4606 change http://review.whamcloud.com/10193 (commit v2_5_59_0-19-g3e8c354) and on b2_5 via http://review.whamcloud.com/11297 (commit v2_5_2_0-32-g0d77156). The HAVE___U32 et.al. #defines were removed from configure on master via LU-5018 change http://review.whamcloud.com/10258 so the only affected version is actually 2.5.3 AFAIK.

It may be that the only work needed is to cherry-pick http://review.whamcloud.com/10258 onto b2_5 for 2.5.4, and in the meantime the patch affecting e2fsprogs can be temporarily reverted.

I've pushed a patch to b2_5 that cherry-picks 10258 (resolving a few conflicts): http://review.whamcloud.com/11840

[Bob Glossman (Inactive)]

I confirm that building b2_5 with http://review.whamcloud.com/11840 added works OK with new e2fsprogs #includes. I agree this is a better solution than backing out the e2fsprogs mod.

[Andreas Dilger]

I've also pushed http://review.whamcloud.com/11901 to temporarily revert the e2fsprogs 5023510dd7 commit from the master-lustre branch. This will allow building e2fsprogs against Lustre 2.5.3 until change 11840 lands and is released.

The 11901 patch is based on the lustre patch series rebased onto the upstream e2fsprogs-1.42.12 release, and will build as 1.42.12.wc1.

[Bob Glossman (Inactive)]

Have verified e2fsprogs with 11901 patch builds OK and generates 1.42.12.wc1 version rpms. With that version installed both master and current b2_5 (without patch 11840) build OK, no build failures in lustre/utils seen. So far have only confirmed this on latest el6.5. Will gradually do other versions and distros as time permits.

[Peter Jones]

Fix for LU-5018 has landed to master and b2_5