[LU-5614] use %kernel_module_package for weak-updates - Whamcloud Community JIRA

Details

Type: Bug
Resolution: Fixed
Priority: Critical
Fix Version/s: Lustre 2.9.0
Affects Version/s: None
Labels:
- llnl
- patch

Severity:
3
Rank (Obsolete):
15704

Description

The correct way to support weak-updates in rpm packages is the vendor defined %kernel_Module_package. This does the right thing on all distributions.

We have used this feature in SGI Lustre for several years, and I plan to work this feature back into the master branch.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

lustre-ldiskfs.files
0.1 kB
16/Sep/14 4:10 AM
lustre-modules.files
0.1 kB
16/Sep/14 4:10 AM
sgi242-simple.spec
9 kB
16/Sep/14 4:10 AM

Issue Links

is blocking

LU-7643 Remove kernel version string from Lustre release field

Resolved

is related to

LU-8343 build failures on SLES

Resolved

LU-8519 build is broken on sles12

Resolved

LU-8383 Wrong tests package dependencies

Resolved

LU-6677 Package lustre correctly for Fedora/RHEL

Closed

is related to

LU-7722 Install/uninstall of Lustre RPMs is slow due to regenerating of initramfs

Open

LU-8377 lustre fail to build on el7

Resolved

LU-3956 Eliminate lbuild's nonstandard build process

In Progress

LU-3953 lustre build system improvments

Closed

mentioned in: Page Loading...

(4 is related to , 1 mentioned in)

Activity

[LU-5614] use %kernel_module_package for weak-updates

Gerrit Updater added a comment - 27/Jun/16 6:55 PM

Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/12063/
Subject: ~~LU-5614~~ build: use %kernel_module_package in rpm spec
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 3b7d27ea22faf1c6d0a37afa724fd9b5c3240322

Gerrit Updater added a comment - 27/Jun/16 6:55 PM Oleg Drokin (oleg.drokin@intel.com) merged in patch http://review.whamcloud.com/12063/ Subject: LU-5614 build: use %kernel_module_package in rpm spec Project: fs/lustre-release Branch: master Current Patch Set: Commit: 3b7d27ea22faf1c6d0a37afa724fd9b5c3240322

James A Simmons added a comment - 21/Jun/16 3:21 AM - edited

I discovered the kabi-whitelist issues with the following error during the build process:

Finding Provides: /usr/lib/rpm/redhat/find-provides
KERNEL ABI COMPATIBILITY WARNING
The kernel ABI reference files (provided by kabi-whitelists) were not found.
No compatibility check was performed. Please install the kABI reference files
and rebuild if you would like to verify compatibility with kernel ABI.

Finding Requires: /usr/lib/rpm/redhat/find-requires

Once I installed kabi-whitelish and the rpms started to work. Also 'rpm ~~qp --release kmod-lustre~~*' prints out

kernel(__wake_up) = 0x642e54ac
kernel(add_wait_queue) = 0x650fb346
kernel(copy_from_user) = 0x3302b500
kernel(default_wake_function) = 0xffd5a395
kernel(kfree) = 0x037a0cba

So its working correct now.

James A Simmons added a comment - 21/Jun/16 3:21 AM - edited I discovered the kabi-whitelist issues with the following error during the build process: Finding Provides: /usr/lib/rpm/redhat/find-provides KERNEL ABI COMPATIBILITY WARNING The kernel ABI reference files (provided by kabi-whitelists) were not found. No compatibility check was performed. Please install the kABI reference files and rebuild if you would like to verify compatibility with kernel ABI. Finding Requires: /usr/lib/rpm/redhat/find-requires Once I installed kabi-whitelish and the rpms started to work. Also 'rpm qp --release kmod-lustre *' prints out kernel(__wake_up) = 0x642e54ac kernel(add_wait_queue) = 0x650fb346 kernel(copy_from_user) = 0x3302b500 kernel(default_wake_function) = 0xffd5a395 kernel(kfree) = 0x037a0cba So its working correct now.

Christopher Morrone (Inactive) added a comment - 21/Jun/16 12:39 AM

In reply to James from gerrit:

I don't modify any RHEL build scripts. Its a simple rpm extract source and build. Nothing fancy. Forcing people to require things like mock to just build client rpms is not going to go over well with sites.

No one is requiring special things like mock. We just require that, when building rpms, the prerequisite BuildRequires must actually be installed. This is a completely normal thing to expect in the rpm packaging world, and it is what all of the behind-the-scenes scripts from the distro assume will be the case. Most people will be able to meet that requirement, I think. To avoid installing the prerequisites, you have to replace system-provided behind-the-scenes scripts with your own. We did that for lbuild, despite how icky I felt doing it. . If you refuse to meet that simple, accepted requirement, you are free to hack up your own solution as well.

I just recommend mock for the places where you don't have access to a system where the BuildRequires are installed, or you don't have the permissions to install those packages. mock is designed exactly for those situations: it sets up a clean base chroot environment, and then installs only the packages required by the source rpm's BuildRequires.

If none of those things will work for people, they can always just download prebuilt binaries. The prebuilt binaries are going to work on a much wider set of kernels once they are no longer tied to one specific kernel version.

Yes, some adaptation of process will be necessary for some people. I know, change is hard. I totally feel your pain. The Lustre community has gotten very used to its oddball way of building and packaging. But that means that Lustre just flat out doesn't build through the standard tools of major distros like Fedora and RHEL. I think the advantage of supporting those standard methods vastly outweighs the short term pain we will feel as we adapt our personal processes to the new packaging methods.

Christopher Morrone (Inactive) added a comment - 21/Jun/16 12:39 AM In reply to James from gerrit: I don't modify any RHEL build scripts. Its a simple rpm extract source and build. Nothing fancy. Forcing people to require things like mock to just build client rpms is not going to go over well with sites. No one is requiring special things like mock. We just require that, when building rpms, the prerequisite BuildRequires must actually be installed. This is a completely normal thing to expect in the rpm packaging world, and it is what all of the behind-the-scenes scripts from the distro assume will be the case. Most people will be able to meet that requirement, I think. To avoid installing the prerequisites, you have to replace system-provided behind-the-scenes scripts with your own. We did that for lbuild, despite how icky I felt doing it. . If you refuse to meet that simple, accepted requirement, you are free to hack up your own solution as well. I just recommend mock for the places where you don't have access to a system where the BuildRequires are installed, or you don't have the permissions to install those packages. mock is designed exactly for those situations: it sets up a clean base chroot environment, and then installs only the packages required by the source rpm's BuildRequires. If none of those things will work for people, they can always just download prebuilt binaries. The prebuilt binaries are going to work on a much wider set of kernels once they are no longer tied to one specific kernel version. Yes, some adaptation of process will be necessary for some people. I know, change is hard. I totally feel your pain. The Lustre community has gotten very used to its oddball way of building and packaging. But that means that Lustre just flat out doesn't build through the standard tools of major distros like Fedora and RHEL. I think the advantage of supporting those standard methods vastly outweighs the short term pain we will feel as we adapt our personal processes to the new packaging methods.

Christopher Morrone (Inactive) added a comment - 20/Jun/16 9:08 PM

Minh said:

we need to install kabi-whitelists rpm don't we?

That is not a requirement for this ticket. It probably won't hurt, but you can track that activity in a separate ticket if you like. It does not need to be linked to this issue.

Christopher Morrone (Inactive) added a comment - 20/Jun/16 9:08 PM Minh said: we need to install kabi-whitelists rpm don't we? That is not a requirement for this ticket. It probably won't hurt, but you can track that activity in a separate ticket if you like. It does not need to be linked to this issue.

Christopher Morrone (Inactive) added a comment - 20/Jun/16 9:04 PM

I'm back from work travel, so I can jump back into the conversation now.

James, I don't believe that kernel-abi-whitelists is required on either RHEL6 or RHEL7. Yes, there will be a warning about it being missing at build time, but that doesn't hurt the packaging process. Clean RHEL6.7 and RHEL7.2 images have been my main testing platforms (with some SLES 12.X on occasion). I checked both of my RHEL images just now, and neither have kernel-abi-whitelists installed. The rpm packages generated under this patch install just fine, and appear to have all required kernel symbols expressed in the rpm --requires section.

By the way, if your packages really lists this:

ksym(snprintf) = 0x9edbecae
ksym(sscanf) = 0x42224298

then there is an additional problem that I didn't remember off the top of my head on the road. Those should not be "ksym" requirements; they should be "kernel" requirements. I would check the package with "rpm -qp --requires" and see how they look. Here are the requirements from the kmod-lustre package on my RHEL6.7
system:

kernel(snprintf) = 0x9edbecae
kernel(sscanf) = 0x42224298

A "kernel" requirement on RHEL means that the RH build system knows that the symbol is provided by the kernel package itself. Any other kernel symbol provided by some external package (e.g. zfs) will be a "ksym" requirement instead of "kernel". So this still implies that Lustre was not built on a system with the prerequisite kernel rpms properly installed.

FYI, SUSE does not have a kernel-abi-whitelists package. They do not even employ the approach of having a symbol whitelist.

https://drivers.suse.com/doc/SolidDriver/SUSE_Kernel_ABI_Stability.html#does-suse-provide-a-kernel-abi-symbol-whitelist

Christopher Morrone (Inactive) added a comment - 20/Jun/16 9:04 PM I'm back from work travel, so I can jump back into the conversation now. James, I don't believe that kernel-abi-whitelists is required on either RHEL6 or RHEL7. Yes, there will be a warning about it being missing at build time, but that doesn't hurt the packaging process. Clean RHEL6.7 and RHEL7.2 images have been my main testing platforms (with some SLES 12.X on occasion). I checked both of my RHEL images just now, and neither have kernel-abi-whitelists installed. The rpm packages generated under this patch install just fine, and appear to have all required kernel symbols expressed in the rpm --requires section. By the way, if your packages really lists this: ksym(snprintf) = 0x9edbecae ksym(sscanf) = 0x42224298 then there is an additional problem that I didn't remember off the top of my head on the road. Those should not be "ksym" requirements; they should be "kernel" requirements. I would check the package with "rpm -qp --requires" and see how they look. Here are the requirements from the kmod-lustre package on my RHEL6.7 system: kernel(snprintf) = 0x9edbecae kernel(sscanf) = 0x42224298 A "kernel" requirement on RHEL means that the RH build system knows that the symbol is provided by the kernel package itself. Any other kernel symbol provided by some external package (e.g. zfs) will be a "ksym" requirement instead of "kernel". So this still implies that Lustre was not built on a system with the prerequisite kernel rpms properly installed. FYI, SUSE does not have a kernel-abi-whitelists package. They do not even employ the approach of having a symbol whitelist. https://drivers.suse.com/doc/SolidDriver/SUSE_Kernel_ABI_Stability.html#does-suse-provide-a-kernel-abi-symbol-whitelist

James A Simmons added a comment - 20/Jun/16 4:58 PM - edited

I found the source of the problem. This change now requires the package kabi-whitelist at least for RHEL6. Do you have this package for RHEL7 and SLES as well? Once I installed kaki-whitelist it appears to work. Well ZFS still gives trouble but it might be a simple case of rebuilding it.

James A Simmons added a comment - 20/Jun/16 4:58 PM - edited I found the source of the problem. This change now requires the package kabi-whitelist at least for RHEL6. Do you have this package for RHEL7 and SLES as well? Once I installed kaki-whitelist it appears to work. Well ZFS still gives trouble but it might be a simple case of rebuilding it.

Minh Diep added a comment - 18/Jun/16 12:58 AM

"Are you ssh into the destination node and then installing the rpm on that node?"
yes

I am not familiar with chroot. For diskless node, I know that LLNL Chaos built the image with the set of rpms, then refresh/boot the node. I'll have to look into chroot more; but I am pretty sure that's the difference and causing the issue.

Can you attach or upload your kmod-lustre-client rpm so I can check its content?

Minh Diep added a comment - 18/Jun/16 12:58 AM "Are you ssh into the destination node and then installing the rpm on that node?" yes I am not familiar with chroot. For diskless node, I know that LLNL Chaos built the image with the set of rpms, then refresh/boot the node. I'll have to look into chroot more; but I am pretty sure that's the difference and causing the issue. Can you attach or upload your kmod-lustre-client rpm so I can check its content?

James A Simmons added a comment - 18/Jun/16 12:47 AM

Are you ssh into the destination node and then installing the rpm on that node? For our systems we create the rpms and chroot into a directory that serves as the root of my image for the diskless test nodes.

[ management server ] -> /export-image/root -> diskless-node:/root
/usr /usr

chroot /export-image
rpm -ivh /tmp/lustre.rpm

Could the chroot environment be causing the ksym issues? In our diskless setup its not really possible to install rpms directly on the test nodes since they are essentially read only. Can you try that setup Minh please.

James A Simmons added a comment - 18/Jun/16 12:47 AM Are you ssh into the destination node and then installing the rpm on that node? For our systems we create the rpms and chroot into a directory that serves as the root of my image for the diskless test nodes. [ management server ] -> /export-image/root -> diskless-node:/root /usr /usr chroot /export-image rpm -ivh /tmp/ lustre .rpm Could the chroot environment be causing the ksym issues? In our diskless setup its not really possible to install rpms directly on the test nodes since they are essentially read only. Can you try that setup Minh please.

Minh Diep added a comment - 17/Jun/16 11:40 PM

I don't see any issue on el6. I built on el6.7 pointing to el6.8 kernel and install kmod lustre on el6.8 system. no issue

[root@onyx-21vm2 lustre-release]# uname -a
Linux onyx-21vm2.onyx.hpdd.intel.com 2.6.32-573.26.1.el6.x86_64 #1 SMP Wed May 4 00:57:44 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
sh autogen.sh
./configure --disable-server --with-linux=/usr/src/kernels/2.6.32-642.1.1.el6.x86_64/
make rpms
scp kmod-lustre-client-2.8.54_61_gcc7a8c9-2.6.32_642.1.1.el6.x86_64.x86_64.rpm onyx-21vm3:/root
scp lustre-client-2.8.54_61_gcc7a8c9-2.6.32_642.1.1.el6.x86_64.x86_64.rpm onyx-21vm3:/root
history

rpm -hiv ./kmod-lustre-client-2.8.54_61_gcc7a8c9-2.6.32_642.1.1.el6.x86_64.x86_64.rpm
[root@onyx-21vm3 ~]# modprobe lustre
LNet: HW CPU cores: 1, npartitions: 1
alg: No test for adler32 (adler32-zlib)
alg: No test for crc32 (crc32-table)
Lustre: Lustre: Build Version: 2.8.54_61_gcc7a8c9
LNet: Added LNI 10.2.4.14@tcp [8/256/0/180]
LNet: Accept secure, port 988
[root@onyx-21vm3 ~]# modinfo lustre
filename: /lib/modules/2.6.32-642.1.1.el6.x86_64/extra/lustre-client/fs/lustre.ko
license: GPL
version: 2.8.54_61_gcc7a8c9
description: Lustre Client File System
author: OpenSFS, Inc. <http://www.lustre.org/>
srcversion: F6CD34A134CF6E45A10B0DB
depends: obdclass,ptlrpc,libcfs,lnet,lmv,mdc,lov
vermagic: 2.6.32-642.1.1.el6.x86_64 SMP mod_unload modversions
[root@onyx-21vm3 ~]# uname -a
Linux onyx-21vm3.onyx.hpdd.intel.com 2.6.32-642.1.1.el6.x86_64 #1 SMP Tue May 31 21:57:07 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@onyx-21vm3 ~]#

I am not sure what's missing. I setup the builder without anything special. installed git, libtool, rpm-build

How did you put the kernel-devel on the builder? cpio or rpm install?

Minh Diep added a comment - 17/Jun/16 11:40 PM I don't see any issue on el6. I built on el6.7 pointing to el6.8 kernel and install kmod lustre on el6.8 system. no issue [root@onyx-21vm2 lustre-release] # uname -a Linux onyx-21vm2.onyx.hpdd.intel.com 2.6.32-573.26.1.el6.x86_64 #1 SMP Wed May 4 00:57:44 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux sh autogen.sh ./configure --disable-server --with-linux=/usr/src/kernels/2.6.32-642.1.1.el6.x86_64/ make rpms scp kmod-lustre-client-2.8.54_61_gcc7a8c9-2.6.32_642.1.1.el6.x86_64.x86_64.rpm onyx-21vm3:/root scp lustre-client-2.8.54_61_gcc7a8c9-2.6.32_642.1.1.el6.x86_64.x86_64.rpm onyx-21vm3:/root history rpm -hiv ./kmod-lustre-client-2.8.54_61_gcc7a8c9-2.6.32_642.1.1.el6.x86_64.x86_64.rpm [root@onyx-21vm3 ~] # modprobe lustre LNet: HW CPU cores: 1, npartitions: 1 alg: No test for adler32 (adler32-zlib) alg: No test for crc32 (crc32-table) Lustre: Lustre: Build Version: 2.8.54_61_gcc7a8c9 LNet: Added LNI 10.2.4.14@tcp [8/256/0/180] LNet: Accept secure, port 988 [root@onyx-21vm3 ~] # modinfo lustre filename: /lib/modules/2.6.32-642.1.1.el6.x86_64/extra/lustre-client/fs/lustre.ko license: GPL version: 2.8.54_61_gcc7a8c9 description: Lustre Client File System author: OpenSFS, Inc. < http://www.lustre.org/ > srcversion: F6CD34A134CF6E45A10B0DB depends: obdclass,ptlrpc,libcfs,lnet,lmv,mdc,lov vermagic: 2.6.32-642.1.1.el6.x86_64 SMP mod_unload modversions [root@onyx-21vm3 ~] # uname -a Linux onyx-21vm3.onyx.hpdd.intel.com 2.6.32-642.1.1.el6.x86_64 #1 SMP Tue May 31 21:57:07 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux [root@onyx-21vm3 ~] # I am not sure what's missing. I setup the builder without anything special. installed git, libtool, rpm-build How did you put the kernel-devel on the builder? cpio or rpm install?

James A Simmons added a comment - 17/Jun/16 9:58 PM - edited

Okay. Lets see if its a RHEL6.X issue. Especially since I don't have access to RHEL7 systems If you manage to build it then the build process must be dependent on something missing on our build machine. If I see this problem other people will to once its released into the wild.

James A Simmons added a comment - 17/Jun/16 9:58 PM - edited Okay. Lets see if its a RHEL6.X issue. Especially since I don't have access to RHEL7 systems If you manage to build it then the build process must be dependent on something missing on our build machine. If I see this problem other people will to once its released into the wild.

Minh Diep added a comment - 17/Jun/16 9:43 PM

it worked on el7. I'll verify on el6 as you did next

[root@onyx-21vm5 lustre-release]# uname -a
Linux onyx-21vm5.onyx.hpdd.intel.com 3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
sh ./autogen.sh
./configure --disable-server --with-linux=/usr/src/kernels/3.10.0-327.18.2.el7.x86_64/
make rpms

scp kmod-lustre-client-2.8.54_61_gcc7a8c9-3.10.0_327.18.2.el7.x86_64.x86_64.rpm root@onyx-24:/root
scp lustre-client-2.8.54_61_gcc7a8c9-3.10.0_327.18.2.el7.x86_64.x86_64.rpm root@onyx-24:/root

[root@onyx-24 ~]# uname -a
Linux onyx-24.onyx.hpdd.intel.com 3.10.0-327.18.2.el7.x86_64 #1 SMP Thu May 12 11:03:55 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@onyx-24 ~]# rpm -hiv ./kmod-lustre-client-2.8.54_61_gcc7a8c9-3.10.0_327.18.2.el7.x86_64.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:kmod-lustre-client-2.8.54_61_gcc7################################# [100%]
[root@onyx-24 ~]# modprobe lustre
[root@onyx-24 ~]# modinfo lustre
filename: /lib/modules/3.10.0-327.18.2.el7.x86_64/extra/lustre-client/fs/lustre.ko
license: GPL
version: 2.8.54_61_gcc7a8c9
description: Lustre Client File System
author: OpenSFS, Inc. <http://www.lustre.org/>
rhelversion: 7.2
srcversion: F6CD34A134CF6E45A10B0DB
depends: obdclass,ptlrpc,libcfs,lnet,lmv,mdc,lov
vermagic: 3.10.0-327.18.2.el7.x86_64 SMP mod_unload modversions
[root@onyx-24 ~]#

Minh Diep added a comment - 17/Jun/16 9:43 PM it worked on el7. I'll verify on el6 as you did next [root@onyx-21vm5 lustre-release] # uname -a Linux onyx-21vm5.onyx.hpdd.intel.com 3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux sh ./autogen.sh ./configure --disable-server --with-linux=/usr/src/kernels/3.10.0-327.18.2.el7.x86_64/ make rpms scp kmod-lustre-client-2.8.54_61_gcc7a8c9-3.10.0_327.18.2.el7.x86_64.x86_64.rpm root@onyx-24:/root scp lustre-client-2.8.54_61_gcc7a8c9-3.10.0_327.18.2.el7.x86_64.x86_64.rpm root@onyx-24:/root [root@onyx-24 ~] # uname -a Linux onyx-24.onyx.hpdd.intel.com 3.10.0-327.18.2.el7.x86_64 #1 SMP Thu May 12 11:03:55 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux [root@onyx-24 ~] # rpm -hiv ./kmod-lustre-client-2.8.54_61_gcc7a8c9-3.10.0_327.18.2.el7.x86_64.x86_64.rpm Preparing... ################################# [100%] Updating / installing... 1:kmod-lustre-client-2.8.54_61_gcc7################################# [100%] [root@onyx-24 ~] # modprobe lustre [root@onyx-24 ~] # modinfo lustre filename: /lib/modules/3.10.0-327.18.2.el7.x86_64/extra/lustre-client/fs/lustre.ko license: GPL version: 2.8.54_61_gcc7a8c9 description: Lustre Client File System author: OpenSFS, Inc. < http://www.lustre.org/ > rhelversion: 7.2 srcversion: F6CD34A134CF6E45A10B0DB depends: obdclass,ptlrpc,libcfs,lnet,lmv,mdc,lov vermagic: 3.10.0-327.18.2.el7.x86_64 SMP mod_unload modversions [root@onyx-24 ~] #

People

Assignee:: Minh Diep

Reporter:: Stephen Champion

Votes:: 1 Vote for this issue

Watchers:: 23 Start watching this issue

Dates

Created:: 12/Sep/14 4:15 AM

Updated:: 06/Jun/17 3:37 PM

Resolved:: 18/Jul/16 9:54 PM