[LU-2391] warning messages for missing symbols when lustre-modules::osd_zfs.ko installed on a system without zfs-modules installed

• Type: Bug
• Resolution: Fixed
• Priority: Blocker
• Fix Version/s: Lustre 2.4.0
• Affects Version/s: Lustre 2.4.0
• Labels: LB
• Severity: 3
• Assignee: Bruno Faccini (Inactive)
• Reporter: Andreas Dilger

Description

On Nov 8, 2012, at 5:54 AM, Prem Sethna Kancherla wrote on lustre-discuss:

While installing lustre 2.3 on Centos 6.3, I am getting a continuous warning messages regarding "osd_zfs.ko" .

How can these warning messages to be removed while installing? They come while installing lustre-modules RPM. Messages are like:

WARNING: /lib/modules/2.6.32-279.5.1.el6_lustre.gb16fe80.x86_64/updates/kernel/fs/lustre/osd_zfs.ko needs unknown symbol zap_cursor_serialize
WARNING: /lib/modules/2.6.32-279.5.1.el6_lustre.gb16fe80.x86_64/updates/kernel/fs/lustre/osd_zfs.ko needs unknown symbol zap_remove
:
:

I think three options exist:

• suppress the warnings from osd_zfs.ko during installation, but this will only work for the lustre-modules RPM, and not for any other module depmod runs from other RPMs or by the user
• delete the osd_zfs.ko module at install time (preferably before depmod is run to produce these error messages). This will prevent the user from using ZFS if it is later installed (probably not a huge chance of problems for users)
• split the osd_zfs.ko module into a separate RPM. This might also avoid licensing issues that users are concerned about due to CDDL.

Issue Links

• is duplicated by: LU-2993 Missing symbol warnings when installing OSD packages
• is duplicated by: LU-2507 Configure and test ZFS using dkms
• is related to: LU-1032 Add dkms support for kernel modules

Comments

[Yang Sheng]

hi, Bruno, Looks like the commit http://review.whamcloud.com/4869 has some defeat:

the osd-zfs & osd-ldiskfs both require lustre-modules. but lustre-modules also require lustre-osd. Please fix it.

[Bruno Faccini (Inactive)]

I am puzzled with the results for the last patch/build auto-tests. Seems that initialization test is (now/recently ?) run separately from others. And as you expected it failed when others have been successful in a separate session! This caused me to 1st think that some of TT-1009 related changes may have been already applied, and to miss the initialization failures.

How can this happen ? With help of a magic "rpm i lustre*" or something else ?

What I can only confirm again is that during manual install of the build :

_ lustre-modules RPM complains about missing "lustre-osd" dependency
_ lustre-modules + lustre-osd-[ldiskfs,zfs] respectively complains about missing "lustre-ldiskfs" or "zfs-modules" dependency.
_ lustre-modules + lustre-osd-[ldiskfs,zfs] + lustre-ldiskfs/zfs-modules is ok.
_ Tests run ok after lustre-modules + lustre-osd-ldiskfs + lustre-ldiskfs install, I did not test for zfs already.

[Bruno Faccini (Inactive)]

Andreas, this is TT-1009 task !! Thus, I can only add/confirm that, after I manually xfered the Malloo build full RPM set, I verified "locally+manually" that the lustre-modules RPM installed without any msgs regarding external+unsatisfied reference, then each of the OSD RPMs installation with either LDISKFS or ZFS separate/required RPM was successful and again without any message, finally I was able to bring up Lustre and run the test-suite.

But I will double-check and verify each RPM content ... And the full test logs !!

[Andreas Dilger]

Maybe I'm missing something, but how did http://review.whamcloud.com/4869 pass testing if extra work is still needed for handling lustre-osd-ldiskfs and lustre-osd-zfs RPM dependencies? I took the fact that the patch testing had passed as a sign that everything is well?

[Bruno Faccini (Inactive)]

These new OSD RPMs require specifc attention from our testing tools. Separate installation, external symbols resolution, ... Andreas already created TT-1009 to address the need.

[Bruno Faccini (Inactive)]

Due to rebase required (no more "build/autoMakefile.am.toplevel" file, now directly inlined within "autoMakefile.am"), causing Maloo/Hudson builds failures, patch #9 submitted including latest changes.

[Bruno Faccini (Inactive)]

Oops, just found that I forgot to comment on patch #6 which seems to work according to my previous comments and expectations.
BTW, patch #7 is out to comply to Prakash last comments.

[Bruno Faccini (Inactive)]

Humm, patch #5 causes Lustre-Client builds failures because the osd-ldiskfs.ko module has not been built but there is an attempt to construct the lustre-osd-ldiskfs sub-package anyway, so I think I need to definitely avoid building OSDs sub-packages during a Client build ...

And lustre-osd-zfs sub-package is not attempted within the even successful Lustre-Server builds including ZFS/SPL packages, so I think I need to revert and construct the lustre-osd-zfs sub-package by default during Lustre-Server builds ...

Seems that actually the only way to detect+fix this (for both Manual/Make and lbuild kinds of builds) is the ugly/is_client way, sorry ... When patches from LU-1199 will land I will align accordingly.

Let see what happen with patch #6.

[Bruno Faccini (Inactive)]

Submitted patch #5 with more of the "flavor" requested by Chris, if I fully understood !!...
BTW, if it does the trick I may only need to compact my post/preun/postun steps for the modules/osd-ldiskfs/osd-zfs sub-packages as SPEC-file macros if possible and not too tricky, to access to Prakash wishes.

[Christopher Morrone (Inactive)]

So what should we decide here, to hold-on until LU-1199 patches are out or finish here and use it as an intermediate solution until LU-1199 is done and then to comply ??

I would prefer to put a hold on this bug until LU-1199 is further along. I'm happy to do this work in LU-1199 actually. It will be much easier to accomplish with the new framework.

The obvious short term fix for Intel's users is to just disable zfs support in the "official" binary builds that Intel puts on its distribution page. Folks that are advanced enough to try Lustre over ZFS this early in the process (before full support appears in a major release) are likely advanced enough to check out the source and build lustre themselves.

[Christopher Morrone (Inactive)]

Its sure that the inheritance is something I have to deal with, but I don't agree with you that using autoconf conditionals breaks rebuilds from src.rpm, it simply and still (I mean as before, since what is failing is auto-remove of the dist tar-ball during "make rpms"!!) can only re-do/build the same build+options ("rpmbuild -bb/bs/ba" works) it comes from, but do we have to expect more from a src.rpm and is this why you feel it is broken ??

Yes, exactly. That is why I feel it is broken. I absolutely expect more from a src.rpm.

We should be able to have a single set of src.rpm files. If you need to build several versions of the same src.rpm file to cover all of the possible options, you are doing it wrong.

Granted, Lustre has been doing it wrong for ten years, but I'm desperately trying to address that in LU-1199. Your patch touches many lines with those autoconf conditionals. I am going to have to remove them all in the very near future to do it "right", and very likely introduce more complicated patch conflicts for me.

Take a look at the direction I am going in patch 3421. Look at how we handled the conditional build of lustre_tests and zfs in the lustre.spec.in. That is where I believe we should be going with the build system. Note that "is_client" is gone. We would just need to add an "ldiskfs" conditional that works like the zfs one and you would be set with what you need to do the osd-zfs and osd-ldiskfs packages.

That patch is not yet ready for everyone to use, but it is already working in LLNL's build farm. That (or very similar) is how we currently rewrite the broken upstream rpm that won't build for us at all.