[LU-5628] Dealing with kernels that have lustre enabled already

• Type: Task
• Resolution: Duplicate
• Priority: Major
• Labels: usk
• Assignee: Dmitry Eremin (Inactive)
• Reporter: Oleg Drokin

Description

Now that the kernels that have lustre (from that staging tree at the moment) included grows and distributions that ship it increase, we need to do something about all the problems this creates for us.

Currently we cannot build our external lustre against such a kernel due to clash in config defines e.g.:

make[1]: Entering directory `/home/green/bk/x86'
  CC [M]  /home/green/git/lustre-current/libcfs/libcfs/linux/linux-tracefile.o
In file included from <command-line>:0:0:
/home/green/git/lustre-current/config.h:26:0: error: "CONFIG_LNET_MAX_PAYLOAD" redefined [-Werror]
 #define CONFIG_LNET_MAX_PAYLOAD LNET_MTU
 ^
In file included from /home/green/bk/linux/include/linux/kconfig.h:4:0,
                 from <command-line>:0:
include/generated/autoconf.h:1571:0: note: this is the location of the previous definition
 #define CONFIG_LNET_MAX_PAYLOAD 1048576
 ^
cc1: all warnings being treated as errors

Once the lustre is moved out of staging tree, another problem will be added - clashing of symbols from lustre includes in the kernel tree (now hidden in secluded staging location so not a problem immediately).

Once the config symbols clash is resolved - the other problem is the clash in module names between in-kernel lustre and out of kernel lustre. Due to in-kernel implementation mostly being geared towards clients and also lacking our debugging aids and such - these modules are not interchangeable really and we need to do something about it too - possibly consider renaming our out of tree modules? This will become a problem once distributions start to enable lustre by default in their kernels (so not a big problem yet too).

Finally there are bound to be symbol clashes between in and out-of kernel lustre modules so we need to do something about that too I suspect, but not sure what so far. A wrapper to change the name a bit?

Issue Links

• is related to: LU-6547 build errors building against recent 4.1-rc1 kernel on Ubuntu 15.04 building linux-tracefile
• is related to: LU-6083 IB with Ubuntu 14.04 client
• is related to: LU-7042 config.h header conflict with OFED 3.18
• is related to: LU-6215 Sync Lustre external tree with lustre linux kernel client

Comments

[Peter Jones]

Fix tracked under LU-7042

[Dmitry Eremin (Inactive)]

Patch landed to master.

[James A Simmons]

I know DVS from Cray is closed source so no one can help much there. Thinking about it it should be up to the external packages to handle the Module.symvers issue themselves. Better way to handle this is sync up libcfs/lnet upstream with master After patch 16418 lands we can close this ticket.

[Dmitry Eremin (Inactive)]

• I agree we need to provide Module.symvers for DVS from Cray. But we don't need to modify any file from kernel.
• I checked the compiled Lustre client for Ubuntu 14.04 works fine and overwrites in-kernel version like OFED package do.

So, this is additional ticket to provide Module.symvers for external programs that would like to link with our modules. I need more info about those programs. How they link now? What API they use?

[Robert Read]

Amazon Linux includes upstream Lustre, and I've heard you can install el6 lustre-client rpm to be able to mount a filesystem. Haven't heard how well the it actually works, though.

[James A Simmons]

For LNet this is the case. In the wild exist external kernel modules that use LNet like DVS from Cray.

Have you tried Intel Lustre on a Distro with upstream Lustre enabled? I have newer Ubuntu versions on the IBM PowerPC but for some mysterious reason Lustre is disabled unlike other Ubuntu systems

[Dmitry Eremin (Inactive)]

Why we need this? Are we assume somebody will link with our modules? What the reason to provide our symbols versions for other?

[James A Simmons]

Patch http://review.whamcloud.com/16418 will resolve the config.h issues with the upstream kernel but for OpenSFS/Intel lustre to run instead of the upstream client we need to modify Module.symvers to replace the symbols form the upstream clients with the master branch much like we do for the OFED external stacks.

[Dmitry Eremin (Inactive)]

The patch http://review.whamcloud.com/16418 will also resolve this. The issue is common with LU-7042.

[Nathaniel Clark]

WORKAROUND:
Ubuntu 14.04 LTS
Linux kernel 3.13

./configure --disable-server --enable-quota --with-max-payload-mb=1

edit config.h to replace ((1)<<20) with 1048576