[LU-6083] IB with Ubuntu 14.04 client

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.9.0
• Labels: patch
• Environment: Ubuntu 14.04 ; 3.13.0-32-generic
• Severity: 3
• Assignee: Nathaniel Clark
• Reporter: Wang Shilong (Inactive)

Description

I compiled Lustre client under Ubuntu 14.04, with network tcp/ip, it works, but
IB didn't.

Download IB driver for ubuntu14(kernel 3.13.0-32-generic) from:
http://www.mellanox.com/page/mlnx_ofed_eula?mtag=linux_sw_drivers&mrequest=downloads&mtype=ofed&mver=MLNX_OFED-2.3-2.0.0&mname=MLNX_OFED_LINUX-2.3-2.0.0-ubuntu14.04-x86_64.iso

Attachment is failed config.log, failed messages firstly come:
"
/usr/src/mlnx-ofed-kernel-2.3/include/linux/compat-2.6.h:17:35: fatal error: linux/compat_autoconf.h: No such file or directory
#include <linux/compat_autoconf.h>
"
And if i skipped this error by remove this including in source file, i still hit following error:

"
configure: error: an external source tree was specified for o2iblnd however I could not find a /usr/src/mlnx-ofed-kernel-2.3/Module.symvers there
"
if i touched a Module.symvers(a little hack) there and compile finished, and i installed these debs, when modprobe lustre with IB, i hit following messages:

[422278.843073] ko2iblnd: Unknown symbol rdma_create_qp (err -22)
[422278.843080] ko2iblnd: disagrees about version of symbol ib_destroy_cq
[422278.843081] ko2iblnd: Unknown symbol ib_destroy_cq (err -22)
[422278.843084] ko2iblnd: disagrees about version of symbol rdma_create_id
[422278.843085] ko2iblnd: Unknown symbol rdma_create_id (err -22)
[422278.843101] ko2iblnd: disagrees about version of symbol rdma_listen
[422278.843103] ko2iblnd: Unknown symbol rdma_listen (err -22)
[422278.843105] ko2iblnd: disagrees about version of symbol rdma_destroy_qp
[422278.843107] ko2iblnd: Unknown symbol rdma_destroy_qp (err -22)
[422278.843113] ko2iblnd: disagrees about version of symbol ib_query_device
[422278.843115] ko2iblnd: Unknown symbol ib_query_device (err -22)
[422278.843119] ko2iblnd: disagrees about version of symbol ib_get_dma_mr
[422278.843120] ko2iblnd: Unknown symbol ib_get_dma_mr (err -22)
[422278.843131] ko2iblnd: disagrees about version of symbol ib_alloc_pd
[422278.843132] ko2iblnd: Unknown symbol ib_alloc_pd (err -22)
[422278.843143] ko2iblnd: disagrees about version of symbol rdma_set_reuseaddr
[422278.843144] ko2iblnd: Unknown symbol rdma_set_reuseaddr (err -22)
[422278.843148] ko2iblnd: disagrees about version of symbol rdma_connect
[422278.843149] ko2iblnd: Unknown symbol rdma_connect (err -22)
[422278.843154] ko2iblnd: disagrees about version of symbol ib_modify_qp
[422278.843156] ko2iblnd: Unknown symbol ib_modify_qp (err -22)
[422278.843168] ko2iblnd: disagrees about version of symbol rdma_destroy_id
[422278.843169] ko2iblnd: Unknown symbol rdma_destroy_id (err -22)
[422278.843174] ko2iblnd: disagrees about version of symbol rdma_accept
[422278.843176] ko2iblnd: Unknown symbol rdma_accept (err -22)
[422278.843189] ko2iblnd: disagrees about version of symbol ib_dealloc_pd
[422278.843190] ko2iblnd: Unknown symbol ib_dealloc_pd (err -22)
[422278.843195] ko2iblnd: disagrees about version of symbol ib_fmr_pool_map_phys
[422278.843196] ko2iblnd: Unknown symbol ib_fmr_pool_map_phys (err -22)
[422278.843452] LNetError: 29038:0:(api-ni.c:1515:lnet_startup_lndnis()) Can't load LND o2ib, module ko2iblnd, rc=256
[422278.853606] LustreError: 29038:0:(events.c:629:ptlrpc_init_portals()) network initialisation failed

Could you guys take a look at this issue.

Issue Links

• is related to: LU-5628 Dealing with kernels that have lustre enabled already
• is related to: LU-1706 Building debian modules for Lustre 2.1.x fails on Ubuntu 10.04.1 LTS
• is related to: LU-5953 lustre[-dkms] needs to automatically account for OFED

Comments

[Wang Shilong (Inactive)]

Sorry for incomplete information, I mean for using git tree built with master branch

[Bob Glossman (Inactive)]

when you say lustre client are you referring to the upstream lustre client that is part of the Ubuntu kernel tree in drivers/staging/lustre, or the lustre client built from the community lustre git tree built on and for Ubuntu?

[Bob Glossman (Inactive)]

don't know if it makes any difference but the kernel version you call out, 3.13.0-32-generic, looks a bit obsolete. The current one in Ubuntu 14.04 is 3.13.0-0.43

[Bob Glossman (Inactive)]

suspect the problem comes from using Mellanox infiniband. Is this really necessary? As far as I know Ubuntu has a perfectly fine inkernel IB stack that is configured in by defauit.

[Peter Jones]

Bob

Could you please review this patch?

Thanks

Peter

[Wang Shilong (Inactive)]

BTW, we need apply this http://review.whamcloud.com/#/c/13129/ for master to make normal compiling pass.