[LU-5851] Handle LU-4606 packaging changes for Lustre Server DKMS RPM

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.7.0
• Type: Bug
• Priority: Critical
• Reporter: Bruno Faccini (Inactive)
• Assignee: Bruno Faccini (Inactive)
• Resolution: Fixed
• Labels: HB
• Severity: 3

Description

LU-4606 has introduced some packaging changes and particularly one that causes a lib/dso to be shipped inside each of the lustre-osd RPMs.
This breaks current packaging of Lustre Server DKMS RPM which presently builds and provides the osd module but also conflicts with the lustre-osd package!
This leads to the Lustre Server DKMS RPM to be unusable due to the necessary lib/dso for the mount command not to be installed.

Issue Links

• is related to: LU-4606 Lustre hard codes libzfs.so.1 in lust...
• is related to: LU-6013 Separate mount helpers for client and...

Comments

[Bruno Faccini (Inactive)]

Patch to implement 1st way (build/install DSO in DKMS RPM itself) to fix hole from DKMS side is at http://review.whamcloud.com/12538.

[Bruno Faccini (Inactive)]

Patch to implement 2nd possible way to fix (ship DSO in lustre/utils generic RPM instead of lustre-osd) is at http://review.whamcloud.com/12550.

[Christopher Morrone]

I don't think either of these approaches are particularly desirable. In the first solution, you are adding user-space libraries to packages that are explicitly for Kernel Modules (The "KM" in "DKMS"). In the second solution, you are probably introducing dependencies on both ldiskfs and zfs to the higher level rpm packages (without necessarily adding the required dependencies to the spec file?) even though our explicit intent was to keep them out of the higher level packages. We wanted to be able to let the sysadmin select one or the other (or both) of the backend filesystems as they choose.

I think it is necessary to take a step back and document the overall build plan that you have in mind. Understanding the use cases and requirements will help us to make more educated recommendations for how to craft a reasonable solution.

[Bruno Faccini (Inactive)]

Humm let say that these 2 first fix attempts/ideas have been a way to start the discussion ... And you should have noticed that I pushed them as fortestonly for instance!

The need is to have the Lustre Server DKMS RPM (mainly coming from work/patches of B.Behlendorf/LLNL) to be still usable, even after an osd-specific lib/dso containing mount hooks has been created and shipped into each of the lustre-osd RPMs (in LU-4606).

May be you will prefer the 3rd one at http://review.whamcloud.com/12589 ?? It attempts to split the lustre-osd RPMs in 2 RPMs, lustre-osd-[zfs,ldiskfs] as originally only shipping the Kernel module and lustre-mount-osd-[zfs,ldiskfs] now shipping the osd-specific lib/dso.

[Christopher Morrone]

Humm let say that these 2 first fix attempts/ideas have been a way to start the discussion ... And you should have noticed that I pushed them as fortestonly for instance!

Yes, I saw the fortestonly. The patches have indeed served amiably as a discussion starter. Here I am conversing with you now. Reviewing the patch was part of engaging in a discussion.

The need is to have the Lustre Server DKMS RPM (mainly coming from work/patches of B.Behlendorf/LLNL) to be still usable

I understand. But when I talk about "taking a step back", I am suggesting that we take a broader look at how the build system works, and the other requirements that need to be maintained while pursuing this goal.

May be you will prefer the 3rd one at "http://review.whamcloud.com/12589"

No, I'm not really in favor of that either. It is not at all clear to me why you would want to build the kernel modules, but not the utilities needed to use the kernel modules. How would you intend to use that change? Be specific. What are the build commands you would use to put together a complete release?

[Christopher Morrone]

Actually the 3rd one is probably the least objectionable. All this bizarre "post-base" junk is making the spec file harder and harder to read. I would really like to see all of that backed out at some point.

I still want to see you write down the use cases. What are the build command that you intend to use to make a working set of packages?

[Bruno Faccini (Inactive)]

Before patch for LU-4606 landed, to be able to have a working Lustre Server (presently zfs-only) installation based on [lustre, spl, zfs]-dkms RPMs use, only a lustre (for the same Lustre version) RPM was required to be installed.

Now that backend/osd-specific hooks have been extracted into different DSOs and are shipped separately as part of their corresponding lustre-osd RPM we need to find a way to install the required DSO but not the lustre-osd module (that is built by lustre-dkms RPM and thus Provides it but also Conflicts with it to prevent dual installs).

So with my 1st patch, where the zfs osd-hooks DSO is built+installed by lustre-dkms, only lustre RPM still needs to be also installed.

With my 2nd patch, where both the ldiskfs and zfs DSOs are shipped as part of lustre RPM, again only lustre RPM still needs to be also installed. But then, their dynamic load by [mkfs,mount,tunefs].lustre tools must not be so strict than presently and allow for failures if some of their load/run-time dependencies are missing, like if spl/zfs modules/libs are not installed, ...

With my 3rd patch, where each osd-hooks DSO is now shipped into a new+separate RPM than the respective/corresponding lustre-osd, they (at least one) will need to be installed in addition to the lustre RPM.

BTW, for both the 2nd/3rd cases/patches, I will also need to re-evaluate and likely have to add/change the necessary Provides/Requires/Conflicts rules in lustre[-dkms].spec files for the concerned lustre-dkms/lustre-modules/lustre-osd/lustre[/lustre-osd-mount] packages.

It is a bit late for me, so I hope to have been flaw-less, enough clear and also to have answered your questions.

[Christopher Morrone]

Option 2, change 12550, seems like the best stop-gap measure as of version Patch Set 6. I marked it -1 but my quibbles are pretty minor and easily addressed.

Longer term, we really need to address some a bigger issue with this code, but it doesn't really need to be addressed in this ticket:

We shouldn't call osd_init() at all for client mounts. Better yet, maybe we should split the server mounts out into a separate mount command, e.g. mount.lustre_server. Or maybe even one for each osd type: mount.losd_zfs, mount.losd_ldiskfs.

[Nathaniel Clark]

Option 2 / 12550 causes a zfs dependency in the base lustre rpm, and I would think that that would cause large headaches for ldiskfs only sites.
Option 3 (seperate user-space rpms with the dso's) seems like the right answer

[Bruno Faccini (Inactive)]

Nathaniel, thanks I think you are right unfortunately and that my testing environment for Option 2 / 12550 was incomplete+wrong and was hiding the libzfs dependency of mount_osd_zfs.so lib. BTW, auto-tests did not trigger it too which may mean that no pure ldiskfs Server install is made during non zfs-reviews?
So since, I easily presume it will not be accepted to revert some of your changes from LU-4606 and switch-back to the method where libzfs is dlopen()'ed in mount_utils_zfs.c instead of being explicitely linked now nor to use, since Option 1 / 12538 has already been abandonned, you are again right there is only Option 3 / 12589 left, unless there is some other and better idea to fix?

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/12589/
Subject: LU-5851 build: split lustre-osd RPMs
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 2aa189790ca95f0f4f1d32fadea3269080091b02

[Bruno Faccini (Inactive)]

Option #1 (http://review.whamcloud.com/12538) and #2 (http://review.whamcloud.com/12550) patches have been abandoned.

[Jodi Levi (Inactive)]

Patches have landed to Master. Please reopen ticket if more work is still needed.