[LU-108] configure --prefix broken for libcfs.ko on master

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.1.0
• Type: Bug
• Priority: Minor
• Reporter: Christopher Morrone
• Assignee: Brian Murrell (Inactive)
• Resolution: Cannot Reproduce
• Severity: 3

Description

On master, "configure --prefix=/some/path" appears to be broken for at least libcfs.ko. I wanted to "make install" into a subdirectory in my home directory, and it failed at this point:

test -z "/bghome/morrone/local/lib" || /bin/mkdir -p "/bghome/morrone/local/lib"
 /usr/bin/install -c -m 644  libcfsutil.a '/bghome/morrone/local/lib'
 ( cd '/bghome/morrone/local/lib' && ranlib libcfsutil.a )
test -z "" || /bin/mkdir -p ""
test -z "/lib/modules/2.6.32-71.el6.ppc64/updates/kernel/net/lustre" || /bin/mkdir -p "/lib/modules/2.6.32-71.el6.ppc64/updates/kernel/net/lustre"
 /usr/bin/install -c -m 644 libcfs.ko '/lib/modules/2.6.32-71.el6.ppc64/updates/kernel/net/lustre'
/usr/bin/install: cannot remove `/lib/modules/2.6.32-71.el6.ppc64/updates/kernel/net/lustre/libcfs.ko': Permission denied
make[4]: *** [install-modulenetDATA] Error 1
make[4]: Leaving directory `/bghome/morrone/lustre/libcfs/libcfs'
make[3]: *** [install-am] Error 2
make[3]: Leaving directory `/bghome/morrone/lustre/libcfs/libcfs'
make[2]: *** [install-recursive] Error 1
make[2]: Leaving directory `/bghome/morrone/lustre/libcfs/libcfs'
make[1]: *** [install-recursive] Error 1
make[1]: Leaving directory `/bghome/morrone/lustre/libcfs'
make: *** [install-recursive] Error 1

Notice it tried to install libcfs.ko into /lib instead of /bghome/morrone/local/lib. (I had specified --prefix=/bghome/morrone/local).

Comments

[Peter Jones]

Brian

Could you please look at this one?

Thanks

Peter

[Brian Murrell (Inactive)]

Chris,

configure's --prefix is not meant to influence where the modules go – only the stuff that would normally be in one's /usr (i/e/ /usr/lib, /usr/bin, etc.)

What you are looking for is typically implemented in Makefiles as "DESTDIR". i.e. "make install DESTDIR=/bghome/morrone/local. This something that is typically created for packaging systems so that they can install into a given tree and package it up without polluting the system tree.

So looking to our lustre.spec.in for guidance, we can see in the %install scriptlet:

make install DESTDIR=$RPM_BUILD_ROOT

So, indeed our Makefiles use DESTDIR to install everything under another directory.

Let me know if that resolves your issue.

[Christopher Morrone]

Well, it doesn't change the fact that "make install" just plain doesn't work when --prefix is used by itself. If nothing else, that seems to break the principle of least surprise.

If --prefix is not functional, then perhaps it should be disabled...but thats pretty standard to autoconf, so that would be pretty odd.

Is there a reason that --prefix can't set DESTDIR globally?

Actually, right now --prefix and DESTDIR don't play well together, which seems to argue
that it really SHOULD be disabled. If I specify the same path with both --prefix and DESTDIR, they will be concatonated for the non-modules, which is completely wrong.

[Brian Murrell (Inactive)]

make install works just fine when configure is given a --prefix because configure's --prefix is only meant to relocate things that would usually go in /usr. An example of what configure's --prefix is for is if you wanted to install the libraries and binaries in /usr/local instead of /usr. Of course, in such a situation, installing the kernel modules in /usr/local/lib/modules would be wrong since the kernel would then not be able to find them.

What you are looking to do, relocate the entire installation to a root other than /, really is exactly what make's DESTDIR is for.

configure's --prefix shouldn't set DESTDIR because they are used for two different things.

Per your example of setting --prefix and DESTDIR to the same thing and having them concatenated for non-modules... this is entirely the expected behavior given what configure's --prefix and make's DESTDIR are for.

To further exemplify, Imagine you wanted your binaries and libraries, etc. in /usr/local/

{bin,lib,...}

and you wanted to stage an installation in "/var/tmp/install" (where you want the modules in /var/tmp/install/lib/modules/$(uname -r)/ because /var/tmp/install acts as the "virtual root" of your installation – i.e. somewhere you could to a "tar -C /var/tmp/install -cvf ... to tar it all up for relocation), you would do:

$ configure --prefix=/usr/local ...
$ make
$ make install DESTDIR=/var/tmp/install

Here you can see how --prefix and DESTDIR are used together to achieve different goals.

[Christopher Morrone]

$ configure --prefix=/usr/local ...
$ make
$ make install DESTDIR=/var/tmp/install

This is broken. I did this:

$ configure --prefix=/bgusr/morrone/local
$ make
$ make install DESTDIR=/bgusr/morrone/local

And things that should appear under /usr wound up here:

/bgusr/morrone/local/bgusr/morrone/local

[Christopher Morrone]

Oh wait, I think I see what you meant by "stage" now.

Seems odd for something that uses configure to work that way. Usually --prefix has the meaning that DESTDIR does in Lustre, and things like --prefix-exec are used to change various parts.

But I suppose it'll do.

[Brian Murrell (Inactive)]

I think you see the difference betwen --prefix and DESTDIR now but just to be pedantic about it...

Seems odd for something that uses configure to work that way. Usually --prefix has the meaning that DESTDIR does in Lustre, and things like --prefix-exec are used to change various parts.

All of this is really not unique to Lustre but actually how most (autoconf based at least) FOSS packages work. If you look in just about any rpm SPEC file you will see this use of DESTDIR in the %install scriptlet. If you look at rpmbuild's output for any such packages (not just lustre) you will also see that rpmbuild typically calls configure explicitly with --prefix=/usr (and typically a ton of other location directives). i.e. Redhat's RPM calls configure with:

+ ./configure ... --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info ...

So using DESTDIR is a workable solution for you?

[Christopher Morrone]

Yes, thanks, that is fine.

[Brian Murrell (Inactive)]

Great. I just wanted to make sure my proposed solution was workable for you before I closed the ticket.