[LU-5953] lustre[-dkms] needs to automatically account for OFED

• Type: Improvement
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.9.0
• Assignee: Nathaniel Clark
• Reporter: Bruno Faccini (Inactive)

Description

Currently our lustre-dkms package does not build against any installed OFED.

Background

Outside of the DKMS world, if a user wants to build Lustre with OFED they first build and install OFED and then using Lustre's configure script, point the Lustre build at the built and installed OFED using the --with-o2ib switch, giving it the path to the OFED installation as an argument.

Problem

This doesn't work with lustre-dkms because there is no way for the user to provide that OFED path to DKMS built (Lustre) modules.

Solution

The nice part is that this solution is general and will be of benefit to users who compile from source in the traditional manner as well as benefiting lustre-dkms users.

The solution that I propose is that the code we currently have to handle --with-o2ib [yes|no|<path_to_ofed>] should look for the OFED installation in it's expected installed location (i.e. when following the official upstream instructions on how to build and install OFED) when given simply a yes option and that OFED should be preferred over in-kernel IB if the argument is simply yes and OFED is installed.

Issue Links

• is related to: LU-5597 Kernel Module.symvers take precendene instead of MOFED provided
• is related to: LU-6083 IB with Ubuntu 14.04 client

Comments

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/20523/
Subject: LU-5953 build: use installed OFED by default with dpkg
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ce8389306ad009f59eb5203260df38ddda16828d

[Gerrit Updater]

Nathaniel Clark ([e-mail]) uploaded a new patch: http://review.whamcloud.com/20523
Subject: LU-5953 build: use installed OFED by default with dpkg
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 53d4d4ba028f2d2df62d836dc43b2ee4ae66ac4e

[Bruno Faccini (Inactive)]

This automatic detection mechanism is intended for IEEL and DKMS Lustre RPMs, as an experienced super-user, again why don't you use the "--with-o2ib=<path>" configure option/way ??

[Wang Shilong (Inactive)]

Generally speaking, i don't think that automatically detecting is a good idea for super users, Bruno, maybe give options to users are better, no?

[Bruno Faccini (Inactive)]

The simplest way to fix this could be to use the "--with-o2ib=<path>" configure option/way! And also to request latest MLNX_OFED versions to continue to provide the %install_path/openib link in their devel rpm?

[James A Simmons]

I'm in the process of setting up a Mellanox stack system so if I run into this issue I will see what I can do to fix it.

[Wang Shilong (Inactive)]

Hello,

At least MLNX_OFED_LINUX-2.4-1.0.4-rhel6.6-x86_64.tar totally breaks your checks..

Please check it and download from following link
http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers

[Bruno Faccini (Inactive)]

Hello Wang,
Thanks to raise this issue, but I am afraid this may be directly linked to MLNX_OFED different (and recent?) packaging than in the OFED RPMs and which was not in the original scope of this ticket and associated patch.

OTH, I remember I have used the "openib" file/link name as a reference because I thought to have found it was kept for historical in the different OFED versions packaging. But seems MLNX_OFED now (I checked it is still true/working in/with MLNX_OFED_LINUX-2.1-*) breaks this behavior?

[Wang Shilong (Inactive)]

Hello,

We failed to build recently with this patch.

It was because we failed following check:
ofed_info | egrep -w 'compat-rdma-devel|kernel-ib-devel|ofa_kernel-devel' | xargs rpm -ql | grep '/openib$'

Here is really output for our building (rhel6.6 with MLNX_OFED_LINUX-2.4-1.0.4-rhel6.6-x86_64):

[root@build01 MLNX_OFED_LINUX-2.4-1.0.4-rhel6.6-x86_64]# ofed_info | egrep -w 'compat-rdma-devel|kernel-ib-devel|ofa_kernel-devel' | xargs rpm -ql | grep '/openib'
/usr/src/ofa_kernel-2.4/ofed_scripts/openib.conf
/usr/src/ofa_kernel-2.4/ofed_scripts/openibd
/usr/src/ofa_kernel-2.4/ofed_scripts/openibd.service
/usr/src/ofa_kernel/default/ofed_scripts/openib.conf
/usr/src/ofa_kernel/default/ofed_scripts/openibd
/usr/src/ofa_kernel/default/ofed_scripts/openibd.service

So maybe you mean '/openibd$' rather than '/openib$' ?

Best regards,
Wang Shilong

[Bruno Faccini (Inactive)]

And this is somewhat its purpose to detect unexpected IB setups ...