[LU-12214] lustre spec file don't provide a right build requirements

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.12.0, Lustre 2.12.6
• Fix Version/s: Lustre 2.15.0
• Type: Bug
• Priority: Minor
• Reporter: Alexey Lyashkov
• Assignee: Alexey Lyashkov
• Resolution: Fixed

Environment

RHEL

• Severity: 3

Description

Lustre spec file have a several issues related to the building.
1) forget to have a build requirements in many cases
2) build failed if without-utils specified
3) several package errors
4) kabi dependence lost caused a build warning.
5) incorrect MPI build settings

Issue Links

• is related to: LU-13711 Typo in libssl dependency
• is related to: LU-9897 Tracking of lustre library improvements.

Comments

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36423
Subject: LU-12214 selinux: Remove concatenating of selinux context
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a5eaa09714a690742d64b23a0c2efb947938348a

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36424
Subject: LU-12214 build: fixes if the name is not just 'lustre'
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 86961d675b501a92df1d56724bb49b1ff5d2e5cc

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36425
Subject: LU-12214 build: fixes for configure --enable-dist
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 0cc0a64d7980ba0e97214941e63cbdaca0f97a6d

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36426
Subject: LU-12214 build: fix build with mofed
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8c60d6144ae01fe97f8fa864c8c39cbfc7c149b0

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36427
Subject: LU-12214 build: fix build with MPI
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 89d88954055bdb75bb43e2ff4163ccf2147fe94a

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36428
Subject: LU-12214 build: add right kernel detect
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d1b917c56878425606e3891764f3b36638efa2c5

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36429
Subject: LU-12214 build: fix build without lustre_utils
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a06537047a4ea96c3614265d10a439c8979303bd

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/36430
Subject: LU-12214 build: fix build with gss enabled
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 7a4c144a41ebc18c547dc3140d30d12814cc33ed

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36423/
Subject: LU-12214 selinux: Remove concatenating of selinux context
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ece2bf7fcf527eb09a99f5d9e9e8c0b4c20af0e2

[Cory Spitz]

mdiep, ofaaland, https://review.whamcloud.com/#/c/36424 has been ready for some time, but it needs another +1. Can you please provide a review? Thanks!

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36424/
Subject: LU-12214 build: fixes if the name is not just 'lustre'
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cfaf0eb92005b77252753c1b9fbbf2ea3c7c1ff5

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36426/
Subject: LU-12214 build: fix build with mofed
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 71297fe48a22e806b77d9f44c858d522ca949e03

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36427/
Subject: LU-12214 build: fix build with MPI
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: f912153f468217cb2f77b69cbd61eea8d9466500

[Shuichi Ihara]

Unfortunately, patch https://review.whamcloud.com/#/c/36427/ was already merged, but after patch merged, 'make rpm' is always asking 'openmpi-devel' as a required package even openmpi is not detected. It should be disabled by default if configure doesn't detect openmpi, shouldn't it?
Also, '--enable-mpitests=no' ignores, then still asking openmpi-devel as an dependency. How can that dependency be disabled? The other situation, the dependency of 'openmpi-devel' only works if people installed openmpi from centos/rhel ISO, but doesn't work if openmpi was installed from MOFED package.

[Alexey Lyashkov]

as i understand - you try to play with configure args - but it completely wrong way with rpm packaging.
as about other question.

1) Once MOFED don't provide an own openmpi-devel rpm - it's looks one more bug in the MOFED packagers.
It mean you can't able to build anything against this install. Can you fill a Mellanox ticket? or I will do it ?

2) %bcond_without mpi .. in spec file, so it mean [rpmbuild | mock] --without=mpi - will block a building mpi tools.
you can change an MPI implementation with --define "mpi_name mpich", if you prefer an MPICH over OpenMPI.

[Shuichi Ihara]

Sure, I would you ask Mellanox fixing of openmpi-devel in MOFED, that's OK. But, "make rpms" still broke after patch 36427. Shouldn't 'configure' detect whehter if openmpi-devel is installed and if it's not installed, I would keep same behavior before patch otherwise people who did't install mpi, claims "make rpms" doesn't work anymore. I know 'rpmbuild lustre.spec' works, but I also would keep "make rpms" is happy because it has been simple and I believe the most of people build rpm with that way other than rpmbuild directly.

[Alexey Lyashkov]

Okey. i will provide a fix for make rpms case.
But as for me, make rpms isn't clean way to build a packages and mock / OBS is better.

[Shuichi Ihara]

Okey. i will provide a fix for make rpms case.

Great, thank you for your understanding.

But as for me, make rpms isn't clean way to build a packages and mock / OBS is better.

Yeah, agreed, but let's also assume non-HPC person who don't know MPI, but want to use Lustre

[Alexey Lyashkov]

> Yeah, agreed, but let's also assume non-HPC person who don't know MPI, but want to use Lustre

It's nothing about HPC. non-HPC person can use mock or make srpm + rpm build or OBS to create a packages.

It probably good time to separate mpi tests into own package. I will look to this.

[Shuichi Ihara]

Since LU-12214 is not closed yet, are you fixing in LU-12214? or should I open new ticket to address them? Again, my points are below

1. Even openmpi-devel is not installed, 'make rpms' shoudn't break and keep same behavior to build rpms rather than 'rpmbuild lustre.spec'
2. '--enable-mpitests=no' should work properly
   It's related to 1), but even disabled mpitests, 'make rpms' still asks openmpi-devel as dependency and 'make rpms' stops.
3. it would be better detection of mpi library rather than just openmpi-devel package checking
   it would be nice that 'configure' should have proper mpi library detection rather than just checking of openmpi-devel rpm. In many cases, people don't install openmpi-devel on client, but compiled from source tarball and installed it to shared directory. (e.g. /apps/openmpi/version/) we don't want to install different openmpi and devel RPM for just lustre build.

[Alexey Lyashkov]

I filled a ticket and started to discus with own friends in Mellanox.
They interested with openmpi right packaging and I think it will be done in short time.

[Shuichi Ihara]

thx, but openmpi in mellanox is not critical. the problem is that after patch 36427 landed, 'make rpms' is broken and lustre can't make rpms even mpi test disabled, that's main problem.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36428/
Subject: LU-12214 build: add right kernel detect
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: c39d269f9d981b988c78136076266279d492f9f8

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37954
Subject: LU-12214 build: fix make rpms
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 723367fa9acb1b116022f379cafda6155ffae2e5

[Alexey Lyashkov]

Can you please check - is this simple change is enough for you, or additional spec file fixes needs.
If needs - please describe an env which need to be tested to build.
Thanks.

[James A Simmons]

BTW I see the same issues with selinux.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37954/
Subject: LU-12214 build: fix make rpms
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 9a085781b34d3dbe46c61db21c2681a0d31136de

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36429/
Subject: LU-12214 build: fix build without lustre_utils
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 15305c3c3fe7894d79c96f4dae0ada6713983376

[James A Simmons]

Will you be pushing a patch that packages lnet into its own RPM. I know cray has carried this for years.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/36430/
Subject: LU-12214 build: fix build with gss enabled
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e1bf37870d8654d24a1e49a1822bb02aa97a6672

[Peter Jones]

Landed for 2.14

[Minh Diep]

This patch caused installation failure. please suggest a suitable deb to install

root@trevis-66vm1:~# apt install libssl1.1+
Reading package lists... Done
Building dependency tree
Reading state information... Done
libssl1.1 is already the newest version (1.1.1-1ubuntu2.1~18.04.5).
0 upgraded, 0 newly installed, 0 to remove and 171 not upgraded.
root@trevis-66vm1:~# apt-get install lustre-client-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
lustre-client-utils : Depends: libssl1.1+ but it is not installable
E: Unable to correct problems, you have held broken packages.

[Andreas Dilger]

It looks like patch https://review.whamcloud.com/36430 "LU-12214 build: fix build with gss enabled" is also causing installation failures on SLES15.1.

This was hit while testing patch https://review.whamcloud.com/39838 "LU-12661 tests: skip sanity 817 for kernel 4.12+" with "Test-Parameters: clientdistro=slies15sp1":
https://testing.whamcloud.com/test_sessions/c673336d-e9c4-4b4a-8d1a-a25044e3c942
https://testing.whamcloud.com/test_sessions/a24570bc-6544-42f8-8089-b5ac5b69103b

Zypper installation of lustre-client lustre-client-tests on node trevis-200vm4 failed
2020-09-09T19:32:27: zypper install output:
Loading repository data...
Reading installed packages...
Resolving package dependencies...
2 Problems:
Problem: nothing provides krb5-libs needed by lustre-client-2.13.55_80_g16b4146-1.x86_64
Problem: nothing provides krb5-libs needed by lustre-client-2.13.55_80_g16b4146-1.x86_64

Problem: nothing provides krb5-libs needed by lustre-client-2.13.55_80_g16b4146-1.x86_64
 Solution 1: do not install lustre-client-2.13.55_80_g16b4146-1.x86_64
 Solution 2: break lustre-client-2.13.55_80_g16b4146-1.x86_64 by ignoring some of its dependencies

Choose from above solutions by number or skip, retry or cancel [1/2/s/r/c/d/?] (c): c
2020-09-09T19:32:27: Zypper install was unsuccessful.

[Andreas Dilger]

The SLES package name is just "krb5", so a patch is needed to make this conditional in the lustre.spec.in file.

[Andreas Dilger]

The most straight forward solution seems to be something like:

 %if %{with gss}
 BuildRequires: krb5-devel openssl-devel
+%if %{_vendor}=="redhat" || %{_vendor}=="fedora"
 Requires: krb5-libs openssl-libs
+%else
+Requires: krb5 openssl
+%endif
%endif

According to rpmfind.net the SLES15sp1 krb5 package provides:

    krb5
    config(krb5)
    krb5(x86-64)
    libgssapi_krb5.so.2()(64bit)
    libgssrpc.so.4()(64bit)
    libk5crypto.so.3()(64bit)
    libkrb5.so.3()(64bit)
    :

while rpmfind.net shows the RHEL8.2 krb5-libs package provides:

    krb5-libs
    config(krb5-libs)
    krb5-libs(x86-64)
    libgssapi_krb5.so.2()(64bit)
    libgssrpc.so.4()(64bit)
    libk5crypto.so.3()(64bit)
    libkrb5.so.3()(64bit)
    :

and the same for RHEL7.x.

It seems possible to change the "package name requires" to "dependency requires" something like the following to work on both distro types:

Requires: libgssapi_krb5.so.2()(64bit) libcrypto.so.10()(64bit)

but I haven't actually tried this. It isn't clear if this would be more portable, or potentially cause other problems in the future. It seems that the krb5-devel and openssl-devel package names are consistent across distros.

[Gerrit Updater]

Alexey Lyashkov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/39972
Subject: LU-12214 build: fix SLES build/install
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d7d6635990f5972b43a217ad37fd9620a2997ff0

[James A Simmons]

For debian I believe the correct package is libssl1.1. 

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/40214
Subject: LU-12214 build: fix suse require krb5
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8dc62655d1e05f5e569f079b01d619d0e7d9d1fb

[Gerrit Updater]

Andreas Dilger ([e-mail]) merged in patch https://review.whamcloud.com/40214/
Subject: LU-12214 build: fix suse require krb5
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: d6c89d28949f384cfd5bc766ea44407b1acdc500

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/39972/
Subject: LU-12214 build: fix SLES build/install
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 253a9e3bcd30bf411c57d77e137ff734eff1edb1

[Peter Jones]

Looks like all the patches have landed for 2.15