Details
-
Improvement
-
Resolution: Fixed
-
Minor
-
Lustre 2.4.1, Lustre 2.5.0
-
11129
Description
Distributions vary in the correct installation directory for kernel modules.
The RHEL standard installation directory is /lib/modules/$(uname -r)/extra, while the SLES standard is /lib/modules/$(uname -r)/updates. For kernel modules packages using the now standard %kernel_modules_package expansion, thsi correct location is indicated by the %kernel_module_package_moddir macro.
Currently, Lustre is only capable of installing modules to the updates/ directory. This is incorrect for RHEL, but has been standing for so long that it should not be changed without some deliberation.
I have a patch to add the capability to support both methods, selecting the destination via configure without changing the default. This creates the option to change it later via the spec, and supports users who desire to make the change now.
Attachments
Activity
The patch landed to master. Which other branch it needs to be?
If kmod2 is using %kernel_module_package, then the work I plan implements kmod2 
The distro macro implementation should do the distro specific packaging - at least, that was the intent of creating %kernel_module_package. I doubt it is all working at intended, and the changes cascade to the build and test infrastructure - but the promise is there.
I found http://rpmfusion.org/Packaging/KernelModules/Kmods2 as well. It does not mention %kernel_module_package directly, but at a glance, the use of kmodtool appears to be drop in compatible with the existing RH macro implementation.
The %kernel_module_package macro is part of the kmods spec, which I'm finding particularly difficult to find any documentation about. Here is about the best that I found:
- http://fedoraproject.org/wiki/Obsolete/KernelModules
- http://driverupdateprogram.com/presentations/DriverUpdateProgramTechnical.pdf
Frankly, I think that patches to convert Lustre to kmods1 or kmods2 packing are going to be a pretty difficult to get into Lustre's upstream repo. Get ready for at least a year-long effort.
That isn't meant to discourage you, just to suggest that perhaps you should target the packaging standard that will be in use at that time.
Fedora already considers kmods1 obsolete. Fedora and RPM Fusion now use kmod2. RHEL pulls from Fedora, so I would not be surprised if RHEL7 uses kmod2, and RHEL7 should be out soon. We could ask around and find out more definitively. I don't know when/if SUSE might follow suit.
One note:
I'd like to see Dmitry's improved version http://review.whamcloud.com/#/c/8065/ shepherded to b2_5 and master, but am happy to abandon my original b2_4 version. I submitted to b2_4 only because that's what I had on hand.
http://review.whamcloud.com/5493 is definitely an improvement, but not what I am trying to accomplish here. This change is only intended to account for differences in normal practices (and tools) in the distributions.
Fragments of wm2 (SuSE) support made it to the Lustre spec, but never really worked correctly.
Our users value the flexibility of updating kernels without a Lustre rebuild and I have a very different build environment. As a result, I have a heavily modified spec which enables weak-modules on RHEL and SLES via %kernel_module_package. I am starting to push some of my changes, but I have to untangle the pieces from my build environment for them to benefit (or at least not harm) anyone else.
The RHEL weak-modules tools do not handle weak-updates links for modules installed to updates/, only to extras/, hence this patch to allow for installing modules to the right place for the distribution.
On the subject of llog : One of the other changes required to use %kernel_module_package is to move all kernel modules into packages which contain only kernel modules. Last I checked, llog was tossed in with the test scripts. I was planning to submit a change to this at some point, either as a tests-modules subpackage or simply shoving it into the modules package.
I'd be interested in kmod2 if the major distributions have plans to support it.
This ticket is not really about the llog_test hack. But that hack, I think, prevents a more simple solution to the problem in this ticket. 5493 is indeed what I was suggesting. After that, we might only need to use a wildcard in the paths in the spec file to address "updates" vs "extra".
Is this patch really just to handle the llog_test hack? In that case I have a patch to handle llog_test in a much better way at http://review.whamcloud.com/#/c/5493. Stephen can you tell me if it address your problem. As for kmod2 it sounds very promising. It exactly what I need for the cray systems. Many a time I wish I could just switch between lustre version without updating rpms.
Moving to the kmod2 package standard would probably be good. It has some problems of its own, but it is in many ways better than what we currently do. For instance, it would allow us to have lustre modules for multiple kernels installed at the same time. One place that would have been useful recently is when we wanted to switch a node back and forth between booting a regular kernel and the .debug version of the kernel. That required juggling rpms which would be unnecessary with kmod2 packaging.
But that would require a rather large effort.
In the mean time, it is kind of a shame that our spec file needs to be aware of this path difference at all. It looks to me like most of the "updates" vs "extra" path issues in the spec file could be addressed by using a wildcard in the path.
The main problem seemed to be the awful "hack to include the llog_test module in lustre-tests". With the recent consolidation of the build systems, it may finally be possible to eliminate those terrible hacks.
If we could then switch to using a path wildcard, the rest of the machinery to tell the spec file about the module path would be unnecessary.
This patch comes up because the RHEL module init tools only work correctly if the modules are installed under extra/. I am trying to inch toward using %kernel_module_package - which eliminates distribution dependent weak-modules problems, but may be incompatible with a relocatable package.
b2_5 would be nice, but warrants more consideration.
As checked in, the patch moves the default location on RH from "updates" to "extras". This is where it should be, but I don't know if the change would cause problems for anyone - so it may not be wanted in the maintenance branch.
It's not a problem for my users.