Details
-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
None
-
None
-
3
-
9223372036854775807
Description
Building Lustre server modules for ubuntu 24.04 were successful, but when built modules were loaded, it failed due to the following errors.
modprobe libcfs modprobe: ERROR: could not insert 'libcfs': Exec format error [ 4159.185743] module libcfs: .gnu.linkonce.this_module section size must match the kernel's built struct module size at run time
I followed steps https://wiki.whamcloud.com/display/PUB/Build+Lustre+MASTER+with+Ldiskfs+on+Ubuntu+20.04.1+from+Git to build lustre modules/binaries.
Although it's a bit old information (for ubuntu 20.04), most of steps still worked for ubuntu24.04.
Also, two patches are still needed for ubuntu 24.04 server build against master.
https://review.whamcloud.com/#/c/fs/lustre-release/+/54216/
https://review.whamcloud.com/#/c/fs/lustre-release/+/55537/
Attachments
Activity
"Shuichi Ihara <sihara@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55711
Subject: LU-18010 build: remove dpatch from dependency
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ae81f85486493729e78455a82b7b78c9b145f9d8
OK, I found much simpler option and can make Lustre server modules/binaries without linux kernel build.
Just link ext4 source codes to /usr/src/linux-headers-6.8.0-31-generic/fs/ext4
ubuntu@src17-c0-n0:~$ wget http://security.ubuntu.com/ubuntu/pool/main/l/linux/linux-source-6.8.0_6.8.0-31.31_all.deb ubuntu@src17-c0-n0:~$ sudo dpkg -i linux-source-6.8.0_6.8.0-31.31_all.deb ubuntu@src17-c0-n0:~$ cd /usr/src/linux-source-6.8.0 ubuntu@src17-c0-n0:~$ sudo tar xjf /usr/src/linux-source-6.8.0/linux-source-6.8.0.tar.bz2 ubuntu@src17-c0-n0:~$ sudo mv /usr/src/linux-headers-6.8.0-31-generic/fs/ext4 /usr/src/linux-headers-6.8.0-31-generic/fs/ext4.orig ubuntu@src17-c0-n0:~$ sudo ln -sf /usr/src/linux-source-6.8.0/linux-source-6.8.0/fs/ext4 /usr/src/linux-headers-6.8.0-31-generic/fs/ext4
Then, go Lustre server build.
ubuntu@src17-c0-n0:~$ git clone git://git.whamcloud.com/fs/lustre-release.git ubuntu@src17-c0-n0:~$ cd lustre-release ubuntu@src17-c0-n0:~$ git checkout -b ubuntu24.04 master ubuntu@src17-c0-n0:~$ git fetch "https://review.whamcloud.com/fs/lustre-release" refs/changes/16/54216/8 && git cherry-pick FETCH_HEAD ubuntu@src17-c0-n0:~$ sh ./autogen.sh ; ./configure --with-o2ib=/usr/src/ofa_kernel/default --enable-server ubuntu@src17-c0-n0:~$ make debs ubuntu@src17-c0-n0:~$ sudo modprobe libcfs
I believe that I've confirmed simplified workable solution.
I will make new page on wiki (Confluence) and describes all steps how to build/run Lustre servers on Ubuntu24.04.
Try apt install --reinstall linux-headers-6.5.0-35-generic or whatever your current kernel version is.
That works only for client. For server build, ext4 related files are still required to apply ldiskfs patches. Above procedures worked in Ubuntu 20.04 at least, but 24.04 can't somehow. I'm trying to find different approach.
Try apt install --reinstall linux-headers-6.5.0-35-generic or whatever your current kernel version is. Its a mismatch issue.
Here is detail steps what I did below.
Select gcc-13
ubuntu@src17-c0-n0:~$ sudo update-alternatives --config gcc There are 3 choices for the alternative gcc (providing /usr/bin/gcc). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/bin/gcc-13 13 auto mode 1 /usr/bin/gcc-11 11 manual mode * 2 /usr/bin/gcc-12 12 manual mode 3 /usr/bin/gcc-13 13 manual mode Press <enter> to keep the current choice[*], or type selection number: 3 update-alternatives: using /usr/bin/gcc-13 to provide /usr/bin/gcc (gcc) in manual mode
build linux kernel in order to build Lustre with running kernel version
ubuntu@src17-c0-n0:~$ uname -r 6.8.0-31-generic ubuntu@src17-c0-n0:~$ wget http://security.ubuntu.com/ubuntu/pool/main/l/linux/linux-source-6.8.0_6.8.0-31.31_all.deb ubuntu@src17-c0-n0:~$ cd /usr/src/ ubuntu@src17-c0-n0:~$ sudo wget http://security.ubuntu.com/ubuntu/pool/main/l/linux/linux-source-6.8.0_6.8.0-31.31_all.deb ubuntu@src17-c0-n0:~$ sudo dpkg -i linux-source-6.8.0_6.8.0-31.31_all.deb ubuntu@src17-c0-n0:~$ tar xjf /usr/src/linux-source-6.8.0/linux-source-6.8.0.tar.bz2 ubuntu@src17-c0-n0:~$ cd linux-source-6.8.0 ubuntu@src17-c0-n0:~$ sed -e 's/^CONFIG_SYSTEM_TRUSTED_KEYS=.*/CONFIG_SYSTEM_TRUSTED_KEYS=""/' -e 's/^CONFIG_SYSTEM_REVOCATION_KEYS=.*/CONFIG_SYSTEM_REVOCATION_KEYS=""/' /boot/config-6.8.0-31-generic > .config ubuntu@src17-c0-n0:~$ touch .scmversion ubuntu@src17-c0-n0:~$ cp Makefile Makefile.orig ubuntu@src17-c0-n0:~$ sed -e 's/^VERSION = .*/VERSION = 6/' -e 's/^PATCHLEVEL = .*/PATCHLEVEL = 8/' -e 's/^SUBLEVEL = .*/SUBLEVEL = 0/' -e 's/^EXTRAVERSION =.*/EXTRAVERSION = -31-generic/' -e 's/^NAME =.*/NAME = Noble Numbat/' Makefile.orig > Makefile ubuntu@src17-c0-n0:~$ make -j $(nproc)
Build lustre against compiled kernel
ubuntu@src17-c0-n0:~$ git clone git://git.whamcloud.com/fs/lustre-release.git ubuntu@src17-c0-n0:~$ cd lustre-release ubuntu@src17-c0-n0:~$ git checkout -b ubuntu24.04 master ubuntu@src17-c0-n0:~$ git fetch "https://review.whamcloud.com/fs/lustre-release" refs/changes/16/54216/8 && git cherry-pick FETCH_HEAD ubuntu@src17-c0-n0:~$ sh ./autogen.sh ; ./configure --with-o2ib=/usr/src/ofa_kernel/default --with-linux=../linux-source-6.8.0/ ubuntu@src17-c0-n0:~$ make
Building lustre worked fine, but it can't be loaded due to same error
ubuntu@src17-c0-n0:~$ sudo insmod ./libcfs/libcfs/libcfs.ko insmod: ERROR: could not insert module ./libcfs/libcfs/libcfs.ko: Invalid module format ubuntu@src17-c0-n0:~$ sudo dmesg [15263.564701] module libcfs: .gnu.linkonce.this_module section size must match the kernel's built struct module size at run time
"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55711/
Subject: LU-18010 build: remove dpatch from dependency
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e4801876a46ad5b3884ccd674e67ef0b3fd67cb9