[LU-1468] Support Compat RDMA for O2IB

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.3.0, Lustre 2.4.0
• Fix Version/s: Lustre 2.4.0, Lustre 2.1.6
• Type: Improvement
• Priority: Blocker
• Reporter: Shuichi Ihara (Inactive)
• Assignee: Minh Diep
• Resolution: Fixed
• Labels: LB

Environment

RHEL6, OFED-3.X

Description

This is an improvement to support OFED-3.2, 3.3, .. with Lustre.
The right now, OFED is moving forward new structure (compat-rdma). The new approach is OFED based on the linux kernel + backports for less kernel revision and few distributions. (RHEL6.2, SLES11SP2, etc.) Here is detail.
http://lists.openfabrics.org/pipermail/ewg/2012-February/017303.html

The benefits are that we can use the lustre with latest Infiniband codes (contains improvements, new features and bug fixes) for the clients and servers. It's not only for the lustre itself, but also we can get improvements for IPoIB, SRP, etc..

Issue Links

• is related to: LU-2975 Build fails on 2.6.32-279.22.1.el6 du...

Comments

[Shuichi Ihara (Inactive)]

patch for master. http://review.whamcloud.com/#change,3011

[Peter Jones]

Ihara

Thanks for the patch!

Minh

Could you please review this patch contribution?

Thanks

Peter

[Minh Diep]

Hi Ihara,

Before diving into the patch, could you confirm that your change is backward compatible to OFED 1.x.x?

[Shuichi Ihara (Inactive)]

Yes, nothing changes for current OFED-1.x.x.
I just added new checker in Makefile and handle if pointed OFED is compat-rdma.
Also, I've tested OFED-1.5.4.1 with patch applied branch. If you can test them with OFED-1.x.x too, that would be great.

[Shuichi Ihara (Inactive)]

tested OFED-3.5-rc4 with patchset10.

[Jodi Levi (Inactive)]

Patch landed to master.

[Minh Diep]

I am porting this to b2_1 which is on later kernel 2.6.32-279.22.1.el6, and I am seeing this broken during build

Type 'make' to build Lustre.
+ make -j 8 -s
Making all in .
In file included from /root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/compat-2.6.h:53,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.h:83,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.c:43:
/root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/compat-2.6.34.h:290:1: error: "rcu_dereference_protected" redefined
In file included from include/linux/tracepoint.h:18,
from /root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/tracepoint.h:19,
from include/linux/module.h:18,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.h:49,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.c:43:
include/linux/rcupdate.h:256:1: error: this is the location of the previous definition
In file included from /root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/compat-2.6.h:59,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.h:83,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.c:43:
/root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/compat-3.0.h:90:1: error: "kfree_rcu" redefined
In file included from include/linux/tracepoint.h:18,
from /root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/tracepoint.h:19,
from include/linux/module.h:18,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.h:49,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.c:43:
include/linux/rcupdate.h:389:1: error: this is the location of the previous definition
make[7]: *** [/root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.o] Error 1
make[6]: *** [/root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd] Error 2
make[5]: *** [/root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds] Error 2
make[4]: *** [/root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet] Error 2
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [_module_/root/lu1468/BUILD/BUILD/lustre-2.1.4] Error 2
make[2]: *** [modules] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.5kT5mc (%build)

patch http://review.whamcloud.com/#change,5688

[Shuichi Ihara (Inactive)]

This is caused due to kernel header(include/linux/rcupdate.h) chagnes on the latest kernel. (2.6.32-279.22.1.el6)
Please see LU-2975.

[Gerrit Updater]

Shuichi Ihara ([e-mail]) uploaded a new patch: http://review.whamcloud.com/12739
Subject: LU-1468 o2iblnd: patch to support OFED-3.5
Project: fs/lustre-release
Branch: b1_8
Current Patch Set: 1
Commit: ddf1a2a25d3d54530587d972d5dec1aec0f63c82