[LU-1468] Support Compat RDMA for O2IB

• Type: Improvement
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.4.0, Lustre 2.1.6
• Affects Version/s: Lustre 2.3.0, Lustre 2.4.0
• Labels: LB
• Environment: RHEL6, OFED-3.X
• Assignee: Minh Diep
• Reporter: Shuichi Ihara (Inactive)

Description

This is an improvement to support OFED-3.2, 3.3, .. with Lustre.
The right now, OFED is moving forward new structure (compat-rdma). The new approach is OFED based on the linux kernel + backports for less kernel revision and few distributions. (RHEL6.2, SLES11SP2, etc.) Here is detail.
http://lists.openfabrics.org/pipermail/ewg/2012-February/017303.html

The benefits are that we can use the lustre with latest Infiniband codes (contains improvements, new features and bug fixes) for the clients and servers. It's not only for the lustre itself, but also we can get improvements for IPoIB, SRP, etc..

Issue Links

• is related to: LU-2975 Build fails on 2.6.32-279.22.1.el6 due to two functions redefined

Comments

[Gerrit Updater]

Shuichi Ihara ([e-mail]) uploaded a new patch: http://review.whamcloud.com/12739
Subject: LU-1468 o2iblnd: patch to support OFED-3.5
Project: fs/lustre-release
Branch: b1_8
Current Patch Set: 1
Commit: ddf1a2a25d3d54530587d972d5dec1aec0f63c82

[Shuichi Ihara (Inactive)]

This is caused due to kernel header(include/linux/rcupdate.h) chagnes on the latest kernel. (2.6.32-279.22.1.el6)
Please see LU-2975.

[Minh Diep]

I am porting this to b2_1 which is on later kernel 2.6.32-279.22.1.el6, and I am seeing this broken during build

Type 'make' to build Lustre.
+ make -j 8 -s
Making all in .
In file included from /root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/compat-2.6.h:53,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.h:83,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.c:43:
/root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/compat-2.6.34.h:290:1: error: "rcu_dereference_protected" redefined
In file included from include/linux/tracepoint.h:18,
from /root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/tracepoint.h:19,
from include/linux/module.h:18,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.h:49,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.c:43:
include/linux/rcupdate.h:256:1: error: this is the location of the previous definition
In file included from /root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/compat-2.6.h:59,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.h:83,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.c:43:
/root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/compat-3.0.h:90:1: error: "kfree_rcu" redefined
In file included from include/linux/tracepoint.h:18,
from /root/lu1468/BUILD/compat-rdma-devel/usr/src/compat-rdma/include/linux/tracepoint.h:19,
from include/linux/module.h:18,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.h:49,
from /root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.c:43:
include/linux/rcupdate.h:389:1: error: this is the location of the previous definition
make[7]: *** [/root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd/o2iblnd.o] Error 1
make[6]: *** [/root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds/o2iblnd] Error 2
make[5]: *** [/root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet/klnds] Error 2
make[4]: *** [/root/lu1468/BUILD/BUILD/lustre-2.1.4/lnet] Error 2
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [_module_/root/lu1468/BUILD/BUILD/lustre-2.1.4] Error 2
make[2]: *** [modules] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.5kT5mc (%build)

patch http://review.whamcloud.com/#change,5688

[Jodi Levi (Inactive)]

Patch landed to master.

[Shuichi Ihara (Inactive)]

tested OFED-3.5-rc4 with patchset10.

[Shuichi Ihara (Inactive)]

Yes, nothing changes for current OFED-1.x.x.
I just added new checker in Makefile and handle if pointed OFED is compat-rdma.
Also, I've tested OFED-1.5.4.1 with patch applied branch. If you can test them with OFED-1.x.x too, that would be great.

[Minh Diep]

Hi Ihara,

Before diving into the patch, could you confirm that your change is backward compatible to OFED 1.x.x?

[Peter Jones]

Ihara

Thanks for the patch!

Minh

Could you please review this patch contribution?

Thanks

Peter

[Shuichi Ihara (Inactive)]

patch for master. http://review.whamcloud.com/#change,3011