[LU-5501] new e2fsprogs causes llverfs build failures

• Type: Bug
• Resolution: Duplicate
• Priority: Major
• Affects Version/s: Lustre 2.5.3
• Environment: e2fsprogs-devel 1.42.11.wc2
• Severity: 3
• Assignee: Jian Yu
• Reporter: Bob Glossman (Inactive)

Description

Some lustre code uses #include files coming from the e2fsprogs rpms, in particular the #include files in /usr/include/ext2fs. With the new version #include files from the latest v1.42.11.wc2 e2fsprogs some strictly user level code that uses those #incudes fail. The only thing that stops this bug from being a Blocker is that it isn't seen when building lustre in master. I think that's due to some of the restructuring of lustre #include that have happened lately in master. It is seen when building in b2_5 and earlier branches.

Pretty sure this is due to changes in e2fsprogs #includes as it is only seen when the new 1.42.11 is installed, not seen with 1.42.9 and earlier. example errors:

gcc -DHAVE_CONFIG_H -I. -I../..  -D__arch_lib__ -D_LARGEFILE64_SOURCE=1 -DLUSTRE
_UTILS=1 -D_FILE_OFFSET_BITS=64 -include /home/bogl/other-lustre/config.h -I/hom
e/bogl/other-lustre/libcfs/include -I/home/bogl/other-lustre/lnet/include -I/hom
e/bogl/other-lustre/lustre/include  -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror 
-MT llverfs.o -MD -MP -MF .deps/llverfs.Tpo -c -o llverfs.o llverfs.c
In file included from /usr/include/e2p/e2p.h:14,
                 from llverfs.c:97:
/usr/include/ext2fs/ext2_fs.h:129: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:137: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:151: error: expected specifier-qualifier-list before ‘__u32’
/usr/include/ext2fs/ext2_fs.h:170: error: expected specifier-qualifier-list before ‘__u32’

For some reason that's not entirely clear to me generic typedefs like __u32 aren't getting pulled in correctly when building strictly userspace code, like in lustre/utils, when #includes from /usr/include/ext2fs are used.

Issue Links

• is related to: LU-5018 e2fsprogs build failed
• is related to: LU-4606 Lustre hard codes libzfs.so.1 in lustre/utils/mount_utils_zfs.c

Comments

[Bob Glossman (Inactive)]

I confirm that building b2_5 with http://review.whamcloud.com/11840 added works OK with new e2fsprogs #includes. I agree this is a better solution than backing out the e2fsprogs mod.

[Andreas Dilger]

It looks like this problem is isolated to b2_5's use of libtool to build the llverfs.c and llverdev.c programs. Testing on b2_4 does not show this problem, even though HAVE___U32 and HAVE_EXT2FS_EXT2FS_H are defined in config.h. That is because the lustre/utils/Makefile uses a "normal" build line that does not implicitly include config.h (which is really intended for the kernel) into userspace programs:

gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverfs llverfs.o
gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverdev llverdev.o  -lblkid

I did my e2fsprogs testing on my test system running master (~2.6.0) and on my production system running b2_4 (where the crazy libtool inclusion didn't exist), so I didn't see this problem. The dependency on libtool was added to master via LU-4606 change http://review.whamcloud.com/10193 (commit v2_5_59_0-19-g3e8c354) and on b2_5 via http://review.whamcloud.com/11297 (commit v2_5_2_0-32-g0d77156). The HAVE___U32 et.al. #defines were removed from configure on master via LU-5018 change http://review.whamcloud.com/10258 so the only affected version is actually 2.5.3 AFAIK.

It may be that the only work needed is to cherry-pick http://review.whamcloud.com/10258 onto b2_5 for 2.5.4, and in the meantime the patch affecting e2fsprogs can be temporarily reverted.

I've pushed a patch to b2_5 that cherry-picks 10258 (resolving a few conflicts): http://review.whamcloud.com/11840

[Bob Glossman (Inactive)]

output of compile of llverfs.c with -E

[Bob Glossman (Inactive)]

include file from e2fsprogs 1.42.11.wc2 present when b2_5 build fails

[Bob Glossman (Inactive)]

Andreas, when I build b2_5 I see the following configure status

checking ext2fs/ext2fs.h usability... no
checking ext2fs/ext2fs.h presence... yes
configure: WARNING: ext2fs/ext2fs.h: present but cannot be compiled
configure: WARNING: ext2fs/ext2fs.h:     check for missing prerequisite headers?
configure: WARNING: ext2fs/ext2fs.h: see the Autoconf documentation
configure: WARNING: ext2fs/ext2fs.h:     section "Present But Cannot Be Compiled"
configure: WARNING: ext2fs/ext2fs.h: proceeding with the preprocessor's result
configure: WARNING: ext2fs/ext2fs.h: in the future, the compiler will take precedence
configure: WARNING:     ## ----------------------------------------- ##
configure: WARNING:     ## Report this to http://bugs.whamcloud.com/ ##
configure: WARNING:     ## ----------------------------------------- ##
checking for ext2fs/ext2fs.h... yes

Note that in spite of the WARNINGS, similar to the ones you show, mine eventually says "checking for ext2fs/ext2fs.h... yes" anyway. My config.h therefore has:

#define HAVE_EXT2FS_EXT2FS_H 1

I will attach the /usr/include/ext2fs/ext2_types-x86_64.h present when the b2_5 build fails.

[Andreas Dilger]

Bob, what does the installed /usr/include/ext2fs/ext2_types-x86_64.h look like? When building b2_5 against e2fsprogs-1.42.11.wc2 I see in the configure output:

checking ext2fs/ext2fs.h usability... no
checking ext2fs/ext2fs.h presence... yes
configure: WARNING: ext2fs/ext2fs.h: present but cannot be compiled
configure: WARNING: ext2fs/ext2fs.h:     check for missing prerequisite headers?
configure: WARNING: ext2fs/ext2fs.h: see the Autoconf documentation
configure: WARNING: ext2fs/ext2fs.h:     section "Present But Cannot Be Compiled"
configure: WARNING: ext2fs/ext2fs.h: proceeding with the compiler's result
configure: WARNING:     ## ----------------------------------------- ##
configure: WARNING:     ## Report this to http://bugs.whamcloud.com/ ##
configure: WARNING:     ## ----------------------------------------- ##
checking for ext2fs/ext2fs.h... no

which is probably why HAVE_EXT2FS_EXT2FS_H is not defined on my test system where I develop e2fsprogs. My config.h does show HAVE___U32 is declared, so the ext2_types.h header should just skip these definitions entirely.

It might be useful to compile with "-E" to see what the actual code looks like after CPP is finished.

[Bob Glossman (Inactive)]

Andreas, strongly suspect the HAVE_EXT2FS_EXT2FS_H is in fact coming from the lustre config.h. Present in your example compile line as: "-include /usr/src/lustre-head/config.h"

That constant is #define'd in both my master build, where the build works, and the b2_5 build, where the build fails.

Can't say why the #includes aren't working as you describe, it seems like they should. I do confirm that backing out the 1 e2fsprogs commit that changes the ext2fs #includes does fix the problem.

[Andreas Dilger]

Could you please explain the actual problem further? The __u32 type is declared in ext2_types.h unless HAVE___U32 is already defined. In the older code, it unconditionally declared the __u32 type, even if it was already declared elsewhere.

This shouldn't affect the llverfs.c and llverdev.c programs, since they don't use __u32 themselves. I'm also wondering where the HAVE_EXT2FS_EXT2FS_H define is coming from? I see that the programs themselves don't include config.h and at least on my system this isn't defined by the build environment:

gcc -DHAVE_CONFIG_H -I. -I../..  -D__arch_lib__ -D_LARGEFILE64_SOURCE=1 -DLUSTRE_UTILS=1 -D_FILE_OFFSET_BITS=64 -include /usr/src/lustre-head/config.h -I/usr/src/lustre-head/libcfs/include -I/usr/src/lustre-head/lnet/include -I/usr/src/lustre-head/lustre/include  -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -MT llverdev.o -MD -MP -MF .deps/llverdev.Tpo -c -o llverdev.o llverdev.c
mv -f .deps/llverdev.Tpo .deps/llverdev.Po
/bin/sh ../../libtool --tag=CC   --mode=link gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -L../../lnet/utils  -o llverdev llverdev.o -lext2fs -lblkid -lkeyutils
libtool: link: gcc -g -Wall -fPIC -D_GNU_SOURCE -g -O2 -Werror -o llverdev llverdev.o  -L/usr/src/lustre-head/lnet/utils -lext2fs -lblkid -lkeyutils

Maybe this has been a longstanding problem, but it is only exposed on some new platform because HAVE_EXT2FS_EXT2FS_H is now defined by the build environment?

[Jian Yu]

Bob verified that installing e2fsprogs built with the above change backed out did make the llverfs and llverdev build problem go away. I'm investigating if there is a way to fix it instead of reverting it.

[Jian Yu]

Hi Andreas,

I saw the following change on lib/ext2fs/ext2_types.h.in, which was committed on Jul 4, 2014:

commit 5023510dd7f0ab40a216d3600ab665e2053cc070
Author: Andreas Dilger <[e-mail]>
Date:   Fri Jul 4 23:34:10 2014 -0400

    blkid,ext2fs: avoid name clash with __u{8,16,32,64}
    
    Try to avoid name clashes with definitions of __u8, __u16, __u32,
    and __u64 in userspace, in case other headers also define these
    types.  Define HAVE___{S,U}{8,16,32,64} preprocessor macros to
    show that these types are already defined.
    
    This would avoid the need to check for _BLKID_TYPES_H in ext2_types.h
    and _EXT2_TYPES_H in blkid_types.h, but since older versions of these
    headers did not use HAVE___U8 et.al. keep these checks around for now.
    
    Report an error if there are no 64-bit types available.  The code
    will not compile if these are not available.
    
    Signed-off-by: Andreas Dilger <[e-mail]>
    Signed-off-by: Theodore Ts'o <[e-mail]>

It looks like this is the culprit that causes the issue in this ticket.

[Jian Yu]

'__u32' is defined in ext2fs/ext2_types.h, which is included in ext2fs/ext2_fs.h. I'll look at what changes were made on ext2fs/ext2_types.h.