[LU-9612] Unable to create rpm-dkms for zfs

• Type: Bug
• Resolution: Done
• Priority: Major
• Affects Version/s: Lustre 2.10.0

Environment

OS SLES12 SP2
kernel-default-4.4.21-69.1.x86_64
nfs-kernel-server-1.3.0-26.3.x86_64
kernel-macros-4.4.21-69.1.noarch
kernel-default-devel-4.4.21-69.1.x86_64
kernel-devel-4.4.21-69.1.noarch
kernel-firmware-20160516git-17.6.noarch
kernel-source-4.4.21-69.1.noarch

Epic/Theme

zfs

• Severity: 2
• Assignee: Malcolm Cowe (Inactive)
• Reporter: Abe (Inactive)

Description

Going through the instructions on http://wiki.lustre.org/Compiling_Lustre zfs build section:

Prepare the build
Run the configure script:
cd $HOME/zfs

1. For RHEL and CentOS only, set the --spec=redhat flag.
   ./configure [--with-spec=redhat] \
   [--with-spl=<path to spl-devel> \
   [--with-linux=<path to kernel-devel>] \
   [--with-linux-obj=<path to kernel obj>]

I issued the following configure command script for suse linux os after build the spl package:

./configure --with-spec=suse --with-spl=/usr/src/spl-0.7.0 --with-linux=/usr/src/linux-4.4.21-69/kernel

It does not create zfs spec under suse folder as it only create these under redhat and generic folders:
config.status: creating rpm/Makefile
config.status: creating rpm/redhat/Makefile
config.status: creating rpm/redhat/zfs.spec
config.status: creating rpm/redhat/zfs-kmod.spec
config.status: creating rpm/redhat/zfs-dkms.spec
config.status: creating rpm/generic/Makefile
config.status: creating rpm/generic/zfs.spec
config.status: creating rpm/generic/zfs-kmod.spec
config.status: creating rpm/generic/zfs-dkms.spec
config.status: creating zfs-script-config.sh
config.status: creating zfs.release
config.status: creating zfs_config.h
config.status: executing depfiles commands
config.status: executing libtool commands

This causes the make rpm-dkms to fail as there are no spec files configured under rpm/suse folder:

make rpm-dkms errors snapshot:cp: cannot stat 'rpm/suse/zfs-dkms.spec': No such file or directory
Makefile:1149: recipe for target 'rpm-local' failed
make[2]: *** [rpm-local] Error 1
make[2]: Leaving directory '/root/zfs'
Makefile:1167: recipe for target 'srpm-common' failed
make[1]: *** [srpm-common] Error 1
make[1]: Leaving directory '/root/zfs'
Makefile:1123: recipe for target 'srpm-dkms' failed
make: *** [srpm-dkms] Error 2

This works for spl packages and not zfs

Issue Links

• is related to: LU-9675 SLES 12 SP2 server build fails when ZFS and LDISKFS OSDs are both enabled

Comments

[Malcolm Cowe (Inactive)]

SUSE provides the SDK add-on – I thought it was one of the standard add-ons for SLES. There might be an ISO download, but I am not sure.

I forget which packages come from the SDK versus the core OS – if you run the above command, it will tell you which packages it couldn't find, if any.

[Abe (Inactive)]

how do you enable the SLE-SDK12 add-on ?

thanks

[Malcolm Cowe (Inactive)]

The following link has the package pre-requisites for compiling ZFS and Lustre:

http://wiki.lustre.org/Compiling_Lustre#SLES_12:_Install_the_Software_Development_Tools

The following command will install the pre-reqs:

sudo zypper install asciidoc automake bc binutils-devel bison bison \
  device-mapper-devel elfutils libelf-devel flex gcc gcc-c++ git \
  glib2-tools glib2-devel hmaccalc libattr-devel libblkid-devel \
  libselinux-devel libtool libuuid-devel lsscsi make mksh ncurses-devel \
  net-tools numactl parted patchutils pciutils-devel perl pesign \
  python-devel rpm-build sysstat systemd-devel tcl tcl-devel tk tk-devel wget \
  xmlto zlib-devel libyaml-devel krb5-devel keyutils-devel net-snmp-devel

You will need to have the SLE-SDK12 add-on enabled. Additional dependencies will be resolved automatically.

[Abe (Inactive)]

cd $HOME/zfs
make pkg-utils pkg-mod ---> returns libzfs compile errors, missing ludev lib
snapshot below:

make[5]: Entering directory '/tmp/zfs-build-root-0K0ri1dq/BUILD/zfs-0.7.0/lib/libzfs'
CC libzfs_changelist.lo
CC libzfs_config.lo
CC libzfs_dataset.lo
CC libzfs_diff.lo
CC libzfs_fru.lo
CC libzfs_import.lo
CC libzfs_iter.lo
CC libzfs_mount.lo
CC libzfs_pool.lo
CC libzfs_sendrecv.lo
CC libzfs_status.lo
CC libzfs_util.lo
CCLD libzfs.la
/usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: cannot find -ludev
collect2: error: ld returned 1 exit status
Makefile:631: recipe for target 'libzfs.la' failed
make[5]: *** [libzfs.la] Error 1
make[5]: Leaving directory '/tmp/zfs-build-root-0K0ri1dq/BUILD/zfs-0.7.0/lib/libzfs'
Makefile:534: recipe for target 'all-recursive' failed
make[4]: *** [all-recursive] Error 1
make[4]: Leaving directory '/tmp/zfs-build-root-0K0ri1dq/BUILD/zfs-0.7.0/lib'
Makefile:707: recipe for target 'all-recursive' failed
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory '/tmp/zfs-build-root-0K0ri1dq/BUILD/zfs-0.7.0'
Makefile:575: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory '/tmp/zfs-build-root-0K0ri1dq/BUILD/zfs-0.7.0'
error: Bad exit status from /tmp/zfs-build-root-0K0ri1dq/TMP/rpm-tmp.NljItE (%build)

RPM build errors:
Bad exit status from /tmp/zfs-build-root-0K0ri1dq/TMP/rpm-tmp.NljItE (%build)
Makefile:1183: recipe for target 'rpm-common' failed
make[1]: *** [rpm-common] Error 1
make[1]: Leaving directory '/root/zfs'
Makefile:1142: recipe for target 'rpm-utils' failed
make: *** [rpm-utils] Error 2

[Malcolm Cowe (Inactive)]

Yes, I would have expected these to have been created as part of the build process, but it depends on the options supplied to the make command for the SPL and ZFS code. 

To create the user space packages, you run this command in both of the SPL and the ZFS directories:

make pkg-utils

This is my build process for SPL and ZFS on SLES 12 SP2:

# SPL
cd $HOME/spl
sh autogen.sh # only needed the first time the code is checked out
./configure --with-linux=/usr/src/linux-4.4.59-92.17
make pkg-utils pkg-kmod
mkdir -p $HOME/releases/zfs-spl
mv $HOME/spl/*.rpm $HOME/releases/zfs-spl
cd $HOME/releases/zfs-spl
sudo rpm -ivh kmod-spl-* spl-0.7.0*.x86_64.rpm

# ZFS
cd $HOME/zfs
./configure --with-linux=/usr/src/linux-4.4.59-92.17
make pkg-utils pkg-kmod
mv $HOME/zfs/*.rpm $HOME/releases/zfs-spl
cd $HOME/releases/zfs-spl
sudo rpm -ivh kmod-zfs-[0-9].*-default-*.x86_64.rpm \
kmod-zfs-devel-[0-9].*.x86_64.rpm \
lib*.x86_64.rpm \
zfs-[0-9].*.x86_64.rpm \
zfs-dracut-[0-9].*.x86_64.rpm

 

[Abe (Inactive)]

The lib,zfs,zfs-dracut rpm are missing, the configure script should have generated these right ?

SBB-02:~/releases/zfs-spl # sudo rpm ivh kmod-spl* spl-.x86_64.rpm kmod-zfs-[0-9].default.x86_64.rpm kmod-zfs-devel-[0-9]..x86_64.rpm lib*.x86_64.rpm zfs-[0-9]..x86_64.rpm zfs-dracut-[0-9]..x86_64.rpm
error: File not found by glob: lib*.x86_64.rpm
error: File not found by glob: zfs-[0-9].*.x86_64.rpm
error: File not found by glob: zfs-dracut-[0-9].*.x86_64.rpm

[Malcolm Cowe (Inactive)]

run the following command:

rpm -qi libzfs2-devel

If the package is not installed then run, from the directory containing the ZFS RPMs you created:

sudo rpm -ivh libzfs2-devel-*.rpm

If that fails, then likely, you need to install all the library packages in the release directory. The simplest way to do that is:

sudo rpm -ivh lib*.rpm

From the output of your commands, the user-space tools might not be installed, either.  

The lustre.org documentation uses the following command to install all the pre-requisite ZFS packages:

cd $HOME/releases/zfs-spl
sudo rpm -ivh kmod-spl-* spl-*.x86_64.rpm \
kmod-zfs-[0-9].*-default-*.x86_64.rpm \
kmod-zfs-devel-[0-9].*.x86_64.rpm \
lib*.x86_64.rpm \
zfs-[0-9].*.x86_64.rpm \
zfs-dracut-[0-9].*.x86_64.rpm

 

[Abe (Inactive)]

Hi Malcom,

your are right, the zfs rpm install is failing, but the spl packages are all installed:

for spl commands used:

sudo zypper install kmod-spl-4.4.21-69-default-0.7.0-rc4.x86_64.rpm kmod-spl-devel-0.7.0-rc4.x86_64.rpm spl-0.7.0-rc4.x86_64.rpm

sudo zypper install kmod-spl-devel-4.4.21-69-default-0.7.0-rc4.x86_64.rpm

SBB-02:~/releases/zfs-spl # rpm -qa | grep spl
kmod-spl-4.4.21-69-default-0.7.0-rc4.x86_64
kmod-spl-devel-0.7.0-rc4.x86_64
libply-splash-core4-0.9.2-29.5.x86_64
libply-splash-graphics4-0.9.2-29.5.x86_64
spl-0.7.0-rc4.x86_64
kmod-spl-devel-4.4.21-69-default-0.7.0-rc4.x86_64

for zfs:

SBB-02:~/releases/zfs-spl # sudo zypper install kmod-zfs-4.4.21-69-default-0.7.0-rc4.x86_64 kmod-zfs-devel-0.7.0-rc4.x86_64 zfs-0.7.0-rc4.src.rpm
Loading repository data...
Reading installed packages...
'kmod-zfs-4.4.21-69-default.x86_64 = 0.7.0-rc4' is already installed.
No update candidate for 'kmod-zfs-4.4.21-69-default-0.7.0-rc4.x86_64'. The highest available version is already installed.
'kmod-zfs-devel.x86_64 = 0.7.0-rc4' is already installed.
No update candidate for 'kmod-zfs-devel-0.7.0-rc4.x86_64'. The highest available version is already installed.
'tmpRPMcache:zfs=0:0.7.0-rc4' not found in package names. Trying capabilities.
No provider of 'tmpRPMcache:zfs=0:0.7.0-rc4' found.
Resolving package dependencies...

Nothing to do.

SBB-02:~/releases/zfs-spl # rpm -qa | grep zfs
kmod-zfs-4.4.21-69-default-0.7.0-rc4.x86_64
kmod-zfs-devel-0.7.0-rc4.x86_64
kmod-zfs-devel-4.4.21-69-default-0.7.0-rc4.x86_64

SBB-02:~/releases/zfs-spl # sudo zypper install zfs-0.7.0-rc4.src.rpm
Loading repository data...
Reading installed packages...
'tmpRPMcache:zfs=0:0.7.0-rc4' not found in package names. Trying capabilities.
No provider of 'tmpRPMcache:zfs=0:0.7.0-rc4' found.
Resolving package dependencies...

SBB-02:~/releases/zfs-spl # ls zfs*
zfs-0.7.0-rc4.src.rpm zfs-dkms-0.7.0-rc4.noarch.rpm zfs-dkms-0.7.0-rc4.src.rpm zfs-kmod-0.7.0-rc4.src.rpm
SBB-02:~/releases/zfs-spl # ls spl*rpm
spl-0.7.0-rc4.src.rpm spl-0.7.0-rc4.x86_64.rpm spl-dkms-0.7.0-rc4.noarch.rpm spl-dkms-0.7.0-rc4.src.rpm spl-kmod-0.7.0-rc4.src.rpm

thanks,
Abe

[Malcolm Cowe (Inactive)]

Abe,

Can you list the SPL and ZFS packages that were installed and the commands used to install them? It looks like the SPL development libraries might not have been installed. The configure script looks for the SPL and ZFS include files, which are in the libzfs2-devel RPM. For example:

vagrant@sl12sp2-b:~rpm -qf /usr/include/libspl
libzfs2-devel-0.7.0-rc4_56_g419c80e6d.x86_64
vagrant@sl12sp2-b:~> rpm -qf /usr/include/libzfs
libzfs2-devel-0.7.0-rc4_56_g419c80e6d.x86_64

The configure script will fail when it cannot find the include files.

[Abe (Inactive)]

that is fine, we are stuck with servers and client rpm build error: Required zfs osd cannot be built due to missing zfs development headers.

Any ideas:

SBB-02:~/lustre-release # ./configure --enable-server --disable-ldiskfs --with-linux=/usr/src/linux-4.4.21-69 --with-linux-obj=/usr/src/linux-4.4.21-69-obj/x86_64/default --with-zfs=/usr/src/zfs-0.7.0 --with-spl=/usr/src/spl-0.7.0

checking zfs source directory... /usr/src/zfs-0.7.0
checking zfs build directory... /usr/src/zfs-0.7.0/4.4.21-69-default
checking zfs source version... 0.7.0-rc4
checking zfs file name for module symbols... Module.symvers
checking user provided zfs devel headers...
checking zfs devel headers... Not Found
configure: error:

Required zfs osd cannot be built due to missing zfs development headers.

thanks,
Abe

[Peter Jones]

Abe

A 2.10.0 tag will be created when the release is finalized. In the meantime, you can just use the tip of master

Peter