Details
-
Improvement
-
Resolution: Fixed
-
Minor
-
Lustre 2.4.1, Lustre 2.5.0
-
11129
Description
Distributions vary in the correct installation directory for kernel modules.
The RHEL standard installation directory is /lib/modules/$(uname -r)/extra, while the SLES standard is /lib/modules/$(uname -r)/updates. For kernel modules packages using the now standard %kernel_modules_package expansion, thsi correct location is indicated by the %kernel_module_package_moddir macro.
Currently, Lustre is only capable of installing modules to the updates/ directory. This is incorrect for RHEL, but has been standing for so long that it should not be changed without some deliberation.
I have a patch to add the capability to support both methods, selecting the destination via configure without changing the default. This creates the option to change it later via the spec, and supports users who desire to make the change now.
Attachments
Activity
http://review.whamcloud.com/5493 is definitely an improvement, but not what I am trying to accomplish here. This change is only intended to account for differences in normal practices (and tools) in the distributions.
Fragments of wm2 (SuSE) support made it to the Lustre spec, but never really worked correctly.
Our users value the flexibility of updating kernels without a Lustre rebuild and I have a very different build environment. As a result, I have a heavily modified spec which enables weak-modules on RHEL and SLES via %kernel_module_package. I am starting to push some of my changes, but I have to untangle the pieces from my build environment for them to benefit (or at least not harm) anyone else.
The RHEL weak-modules tools do not handle weak-updates links for modules installed to updates/, only to extras/, hence this patch to allow for installing modules to the right place for the distribution.
On the subject of llog : One of the other changes required to use %kernel_module_package is to move all kernel modules into packages which contain only kernel modules. Last I checked, llog was tossed in with the test scripts. I was planning to submit a change to this at some point, either as a tests-modules subpackage or simply shoving it into the modules package.
I'd be interested in kmod2 if the major distributions have plans to support it.
This ticket is not really about the llog_test hack. But that hack, I think, prevents a more simple solution to the problem in this ticket. 5493 is indeed what I was suggesting. After that, we might only need to use a wildcard in the paths in the spec file to address "updates" vs "extra".
Is this patch really just to handle the llog_test hack? In that case I have a patch to handle llog_test in a much better way at http://review.whamcloud.com/#/c/5493. Stephen can you tell me if it address your problem. As for kmod2 it sounds very promising. It exactly what I need for the cray systems. Many a time I wish I could just switch between lustre version without updating rpms.
Moving to the kmod2 package standard would probably be good. It has some problems of its own, but it is in many ways better than what we currently do. For instance, it would allow us to have lustre modules for multiple kernels installed at the same time. One place that would have been useful recently is when we wanted to switch a node back and forth between booting a regular kernel and the .debug version of the kernel. That required juggling rpms which would be unnecessary with kmod2 packaging.
But that would require a rather large effort.
In the mean time, it is kind of a shame that our spec file needs to be aware of this path difference at all. It looks to me like most of the "updates" vs "extra" path issues in the spec file could be addressed by using a wildcard in the path.
The main problem seemed to be the awful "hack to include the llog_test module in lustre-tests". With the recent consolidation of the build systems, it may finally be possible to eliminate those terrible hacks.
If we could then switch to using a path wildcard, the rest of the machinery to tell the spec file about the module path would be unnecessary.
This patch comes up because the RHEL module init tools only work correctly if the modules are installed under extra/. I am trying to inch toward using %kernel_module_package - which eliminates distribution dependent weak-modules problems, but may be incompatible with a relocatable package.
It's not the same case. This patch just fix OS specific requirements for installation of additional kernel modules. The main kernel modules directory become the same (/lib/modules/<kernel version>/).
Probably having ability to install in any place also can be beneficially but should be tracked separately.
I ran it this issue with the cray environment some time ago and posted a patch that allowed installing modules in a more non-standard way. At the time it was looked at as Cray specific but we now see that is not the case. For Cray systems the lustre modules are stored in some special /opt directory. Tho that might be more related to ticket LU-3958.
http://review.whamcloud.com/#/c/8013/
This is against b2_4. It does not apply cleanly to master because of the ldiskfs shuffle. It should be an easy port to master, as only file locations are changed - but I don't have a master build environment to test the changes, so am submitting the b2_4 changes I am using.
One note:
I'd like to see Dmitry's improved version http://review.whamcloud.com/#/c/8065/ shepherded to b2_5 and master, but am happy to abandon my original b2_4 version. I submitted to b2_4 only because that's what I had on hand.