[LU-3389] Lustre b2_1 build failed on RHEL6.4 with OFA IB stack

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.4.1, Lustre 2.5.0
• Affects Version/s: Lustre 2.1.6
• Labels: mn1
• Environment: Distro: RHEL6.4; Network: OFA IB
• Severity: 3
• Assignee: Minh Diep
• Reporter: Jian Yu

Description

After http://review.whamcloud.com/5504 was landed on Lustre b2_1 branch, build on RHEL6.4 distro with OFA IB stack has been failing:

http://build.whamcloud.com/job/lustre-b2_1/198/
http://build.whamcloud.com/job/lustre-b2_1/203/

The OFA version is 1.5.4.

Comments

[Wally Wang (Inactive)]

LU-4266 fixed our problem.

[Wally Wang (Inactive)]

Lustre 2.5 server build fails with OFED 3.5-2. It appears to be unnecessarily providing -DCONFIG_COMPAT_RHEL_6_4 and -I${O2IBPATH}/include to the ldiskfs build. And we probably don't need O2IBPATH for any modules not related to O2IB.

[   96s] make[3]: Leaving directory `/home/abuild/rpmbuild/BUILD/cray-lustre/lustre'
[   96s] /usr/bin/make CC="gcc"  -C /usr/src/linux-2.6.32-358.el6_1.0000.7630-obj/x86_64/cray_ari_s_cos	     \
[   96s] 	-f /home/abuild/rpmbuild/BUILD/cray-lustre/build/Makefile LUSTRE_LINUX_CONFIG=/usr/src/linux-2.6.32-358.el6_1.0000.7630-obj/x86_64/cray_ari_s_cos/.config \
[   96s] 	LINUXINCLUDE='-DCONFIG_COMPAT_RHEL_6_4 -I/usr/src/kernel-modules-ofed/x86_64/cray_ari_s_cos/include -I$(srctree)/arch/$(SRCARCH)/include -I$(srctree)/arch/$(SRCARCH)/include/generated -Iinclude $(if $(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) -I$(srctree)/arch/$(SRCARCH)/include/uapi -Iarch/$(SRCARCH)/include/generated/uapi -I$(srctree)/include/uapi -Iinclude/generated/uapi -include include/linux/autoconf.h' \
[   96s] 	M=/home/abuild/rpmbuild/BUILD/cray-lustre -o tmp_include_depends -o scripts -o \
[   96s] 	include/config/MARKER modules
[   96s] make[3]: Entering directory `/usr/src/linux-2.6.32-358.el6_1.0000.7630-obj/x86_64/cray_ari_s_cos'
[   96s] /usr/bin/make -C ../../../linux-2.6.32-358.el6_1.0000.7630 O=/usr/src/linux-2.6.32-358.el6_1.0000.7630-obj/x86_64/cray_ari_s_cos/. modules
[   97s]   CC [M]  /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/acl.o
[   97s]   CC [M]  /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/balloc.o
[   97s]   CC [M]  /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/bitmap.o
[   97s]   CC [M]  /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/block_validity.o
[   97s]   CC [M]  /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/dir.o
...
...
[   99s]   CC [M]  /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/super.o
[   99s] In file included from /usr/src/linux-2.6.32-358.el6_1.0000.7630/include/trace/ftrace.h:441,
[   99s]                  from /usr/src/linux-2.6.32-358.el6_1.0000.7630/include/trace/define_trace.h:74,
[   99s]                  from /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/trace/events/ldiskfs.h:905,
[   99s]                  from /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/super.c:57:
[   99s] /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/trace/events/ldiskfs.h: In function 'ftrace_profile_enable_ldiskfs_free_inode':
[   99s] /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/trace/events/ldiskfs.h:18: error: implicit declaration of function 'register_trace_ldiskfs_free_inode'
[   99s] In file included from /usr/src/linux-2.6.32-358.el6_1.0000.7630/include/trace/ftrace.h:441,
[   99s]                  from /usr/src/linux-2.6.32-358.el6_1.0000.7630/include/trace/define_trace.h:74,
[   99s]                  from /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/trace/events/ldiskfs.h:905,
[   99s]                  from /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/super.c:57:
[   99s] /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/trace/events/ldiskfs.h: In function 'ftrace_profile_disable_ldiskfs_free_inode':
[   99s] /home/abuild/rpmbuild/BUILD/cray-lustre/ldiskfs/trace/events/ldiskfs.h:18: error: implicit declaration of function 'unregister_trace_ldiskfs_free_inode'
[   99s] In file included from /usr/src/linux-2.6.32-358.el6_1.0000.7630/include/trace/ftrace.h:441,
[   99s]                  from /usr/src/linux-2.6.32-358.el6_1.0000.7630/include/trace/define_trace.h:74,
[...

[Peter Jones]

Landed for 2.4.1 and 2.5. Will land to b2_1 if/when work on a 2.1.7 starts

[Shuichi Ihara (Inactive)]

http://review.whamcloud.com/7216 for b2_4
http://review.whamcloud.com/7217 for b2_1

[Shuichi Ihara (Inactive)]

I ported patch for b2_1 and b2_4 and will post them shortly.

[Minh Diep]

b2_1 and b2_4 are now building with kernel 2.6.32-358.11.1 which has no external OFED support (or may be broken even ok to build).

[Jian Yu]

Or is more work needed in this ticket for OFED?

1) the patches need to be landed on Lustre b2_1 and b2_4 branches
2) OFA builds need to be enabled on Jenkins

[Jodi Levi (Inactive)]

Now that the 2 patches have landed, can this ticket be closed? Or is more work needed in this ticket for OFED?

[Shuichi Ihara (Inactive)]

The patches work to build OFED-3.5-1 against RHEL6.4, but Mellanox OFED has different Macro name today.
Here is quick description to build MLNX_OFED_LINUX-2.0-2.0.5 against the latest lustre patched kernel based on RHEL6.4. Today, we need adding macro name that we want to avoid due to OFED doesn't have automatically kernel verion detection. I asked Mellanox and they seems to be trying fix in the future release.

# EXTRA_LNET_INCLUDE="-DCONFIG_COMPAT_IS_PHYS_ID_STATE -DCONFIG_COMPAT_IS_PCI_PHYSFN \
-DCONFIG_COMPAT_IS_KSTRTOX -DCONFIG_COMPAT_IS_BITOP \
-DCONFIG_COMPAT_NETLINK_3_7 -DCONFIG_COMPAT_IS_IP_TOS2PRIO \
-DCONFIG_COMPAT_IS_NETIF_RSS_QUEUES -DCONFIG_COMPAT_IS_NOOP_LLSEEK \
-DCONFIG_COMPAT_IS_SIMPLE_OPEN -DCONFIG_COMPAT_RCU \
-DCONFIG_COMPAT_HAS_NUM_CHANNELS -DCONFIG_COMPAT_ETHTOOL_OPS_EXT" \
./configure --with-o2ib=/usr/src/ofa_kernel

# EXTRA_LNET_INCLUDE="-DCONFIG_COMPAT_IS_PHYS_ID_STATE -DCONFIG_COMPAT_IS_PCI_PHYSFN \
-DCONFIG_COMPAT_IS_KSTRTOX -DCONFIG_COMPAT_IS_BITOP \
-DCONFIG_COMPAT_NETLINK_3_7 -DCONFIG_COMPAT_IS_IP_TOS2PRIO \
-DCONFIG_COMPAT_IS_NETIF_RSS_QUEUES -DCONFIG_COMPAT_IS_NOOP_LLSEEK \
-DCONFIG_COMPAT_IS_SIMPLE_OPEN -DCONFIG_COMPAT_RCU \
-DCONFIG_COMPAT_HAS_NUM_CHANNELS -DCONFIG_COMPAT_ETHTOOL_OPS_EXT" make rpms

[Shuichi Ihara (Inactive)]

http://review.whamcloud.com/6448

patch to build master on RHEL6.4 + OFED-3.5.1.