[LU-688] A patch made to b1_8 has exposed an issue which shows itself as an issue on the Ubuntu build

• Type: Bug
• Resolution: Won't Fix
• Priority: Minor
• Affects Version/s: Lustre 1.8.7, Lustre 1.8.6
• Severity: 3
• Assignee: Brian Murrell (Inactive)
• Reporter: Chris Gearing (Inactive)

Description

A patch made to b1_8 has exposed an issue which shows itself as an issue on the Ubuntu build. No other builds are affected.

The error only under Ubuntu is
make[6]: *** No rule to make target `auster.sh', needed by `all-am'. Stop.

The strange thing is the source does not [appear] to contain the word 'auster.sh', perhaps this is a Lustre issue, but starting with Jenkins seems to make most sense.

// From the Skype Chat.
I'm trying to understand is the ubuntu failure on this build; http://build.whamcloud.com/job/lustre-reviews/2091/ which only fails for ubuntu.

The error only under Ubuntu is
make[6]: *** No rule to make target `auster.sh', needed by `all-am'. Stop.

The strange thing is the source does not [appear] to contain the word 'auster.sh'

At present I'm trying to replicate localling, but figured whilst I do that I should ask you folks for ideas.
Michael MacDonald 13:39
sounds like a 'make dist' problem

after a checkout from git, a source tarball is created via 'make dist', and the tarball is used to feed the package production process

ah, the commit message: LU-261 Renamed auster.sh to be auster making it like b2_1

there ya go

oh, heh. you're the author of that commit.

and now i understand what you mean re: the string "auster.sh"

sorry, started talking before i'd started thinking... hmm. this is weird. when i pull that changeset and grep i don't find auster.sh either.

i wonder if it has something to do with the buildstep stuff in the job configuration... looking at that now

i don't see any hard-coded instances of 'auster.sh' there either

i dunno, probably best to ping brian, as he knows the ubuntu packaging best.
chrisgearing 15:45
Brian: Hello.
Brian J. Murrell 15:47
chrisgearing: hi
chrisgearing 15:56
Do you have any input into the last 2 screens of thread, we see an issue with ubuntu builds failing, but others succeeding. It is the last thread on this chat.
Brian J. Murrell 16:08
yeah. i'd guess it's some issue with the patch de-application/application.

hrm. can i not checkout the branch that gerrit creates with:
git checkout c20175594a8d5cbddd22787b79e3d52478bb8612
?

nm

chrisgearing: somewhere somebody's broken b1_8 so that autogen.sh does not work on a post-"make dist"ed tarball source pool

Checking for a complete tree...
Your tree seems to be missing libcfs.
Please read README.lustrecvs for details.

chrisgearing: it seems unbelievable though, that the change that's causing that problem went in 3 years ago and this issue is only just now being discovered. perhaps the ability to use autogen.sh on a tarball extracted source tree has just never been needed. but even in our own building, i'd expect this to have been needed before now.

chrisgearing: probably somebody more familiar with the history of b1_8 needs to figure out the right solution. you should file a ticket. feel free to CC me on it.
chrisgearing 21:04
What is the description of the ticket, I don't understand the problem you are describing.
Johann Lombardi: ok
Johann Lombardi: i am not sure i get it
Johann Lombardi: "Checking for a complete tree...
Your tree seems to be missing libcfs.
Please read README.lustrecvs for details."
Johann Lombardi: libcfs has moved in master
Johann Lombardi: but not on b1_8
Johann Lombardi: on b1_8, it is still in lnet/libcfs
Johann Lombardi: is that the problem?
Johann Lombardi: i think rread was the one handling the libcfs cleanup
chrisgearing: The build fails only on Ubuntu because it can't find auster.sh, but the words auster.sh do not exist anywhere in the source tree. I cannot understand therefore what could possibly be going on.
Johann Lombardi: hm
chrisgearing: It builds on all other distributions.

My change was rename auster.sh to auster and change the makefile which is the only place the word auster.sh existed.

Brian has lost me in his assessement [because of my lack of knowledge]

I don't even know what to put in a ticket. I can't raise it agsint b1_8 because the bug doesn't occur until my patch is pushed - which can't be pushed!
Johann Lombardi: ok, give me 5min to checkout the branch and have a look around
Johann Lombardi: btw, in the build log, i don't see anything wrong with libcfs
Johann Lombardi: it clearly fails because of something around auster.sh, so i'm not sure why brian thinks that this might somehow be related to libcfs
Johann Lombardi: chris: if you get the full log of the build and search for auster
Johann Lombardi: it says:
Johann Lombardi: reverting patch 0020-LU-261-Renamed-auster.sh-to-be-auster-making-it-like from ./ ... ok.
Johann Lombardi: so somehow the debian build system has reverted the patch
Johann Lombardi: and then later:
Johann Lombardi: dpkg-source: warning: ignoring deletion of file lustre/tests/auster
Johann Lombardi: and then later
Johann Lombardi: lustre/quota/quota_interface.c
lustre/tests/Makefile.am
lustre/tests/Makefile.in
lustre/tests/auster.sh
Johann Lombardi: ah, and then the patch is applied again (gosh)
Johann Lombardi: debian/patches/0020-LU-261Renamed-auster.sh-to-be-auster-making-it-like.dpatch:-- a/lustre/tests/Makefile.am
Checking for a complete tree...
Your tree seems to be missing libcfs.
Please read README.lustrecvs for details.
dh_testdir
Johann Lombardi: and we have the error pointed out by brian, ouf ...
chrisgearing: It reverts a bunch of patches, I wonder how it decides on which ones.
Johann Lombardi: Johann Lombardi is looking at autogen.sh
Johann Lombardi: i have no idea
Johann Lombardi: so it checks for build/libcfs/lnet/lustre dir to be there
Johann Lombardi: but libcfs is in lnet in 1.8
Johann Lombardi: although i think we still have the dir in the root, although not used
Johann Lombardi: in any case, your patch is totally unrelated to this
Johann Lombardi: chris: let me try to repost your patch with libcfs removed from REQUIRED_DIRS
Johann Lombardi: ok, new patch pushed. let's wait for the build result.
Johann Lombardi: although this still makes no sense
chrisgearing: ta.
Johann Lombardi: hm
Johann Lombardi: Applying patch 0018-LU-607-Avoid-race-between-ldlm_pools_shrink-and-ldlm to ./ ... ok.
applying patch 0019-LU-646-port-bz23485-clarification-of-lustre-fsync-be to ./ ... ok.
applying patch 0020-LU-261-Renamed-auster.sh-to-be-auster-making-it-like to ./ ... ok.
touch patch-stamp

1. see if any patches requires us to run autogen
2. (for a distribution release tarball, it is expected that i
   Johann Lombardi: i guess your patch required autogen to be rerun because you modified a Makefile.am
   Johann Lombardi: this has probably not happen on b1_8 for a while ....
   chrisgearing: Ignoring everything else are you saying that the build system does not build from the ground up?
   Johann Lombardi: let me see when this libcfs was added to REQUIRED_DIRS
   Johann Lombardi: ah, yes, libcfs was indeed added by Robert when he worked on the libcfs cleanup
   Johann Lombardi: chrisgearing: well, i'm not sure, i just try to find a rational ....
   Johann Lombardi: and the thing is there is a libcfs dir in 1.8
   Johann Lombardi: although not used
   Johann Lombardi: so the test should really pass ....
   Johann Lombardi: tbh, i don't really understand how our build system works
   Johann Lombardi: i think only brian can tell us why libcfs has suddenly disappear (or was not copied) to the tree used by the ubuntu build
   Johann Lombardi: it seems like ubuntu build has been successful
   Johann Lombardi: so i guess we could land the patch since checking for libcfs does not make sense. However, it would be great if Brian could tell us what is going on
   chrisgearing: Brian said soomething about the tree coming from a tar file. If I understood correctly is it possible that a cloan from git does create the empty libcfs dir, but the tar extraction does not.