Details
-
Improvement
-
Resolution: Fixed
-
Minor
-
None
-
None
-
5154
Description
I think that we need to take a step back and address some fundamental problems with the lustre spec file.
My number one complaint is that lustre source rpms are not buildable (using rpmbuild) in any sane fashion. This has been a constant source of work for folks here at Livermore, because our entire automated build infrastructure is built around the concept of taking source rpms as input and generating binary rpms.
Because the lustre srpms are not buildable, no one tries to do that (except for us). So we fix things for us, but then almost every time the spec file is touched, it breaks something for us again.
I think we need to get on the same page, and finally clean up the lustre spec file. I think the requirements here are:
1) lustre srpm must be rebuildable
2) spec file must take sane options, to allow configuration of lustre at srpm rebuild time
3) must make no attempt to parse options out of configure_args
I think the greatest area of pain will be dealing with "make rpms". The current way that the build system passes option from the initial configuration step down through make rpms and into the spec file is, at best, jerry-rigged. Passing a single giant "configure_args" define into rpmbuild and then parsing out options to set rpm variables is just backwards from the way that that an rpm spec file SHOULD work.
A spec file should take options (rpmbuild -with/-without options when applicable), and then assemble a configure line based on THAT.
Another example of the spec file having it the wrong-way around is "is_client". In order to perform a build that leaves out the server code and rpm dependencies, you set the package NAME! Thats crazy, for not the least reason that now no one can choose to name the package differently when they want a client-only build.
A better approach is to have an rpmbuild "--without servers" option. That sets a variable in the spec file. The spec file uses that flag to conditionally include Requires directives and such, and the lustre package name is changed to "lustre-client" IFF the lustre_name variable has not already been set. This would allow me, for instance, to generate a set of binary rpms with a package name of "lustre-orion", and also make it a client-only build. I cannot accomplish that with the current spec file.
So now the big question: Do we need "make rpms" at all? Where is it really needed? I suspect that Whamcloud's build farm uses it, but I think that you guys should really switch to generating binary rpms from a single source rpm once we get that working reliably.
If we do, I think there are ways that we can make that work with a spec file and source rpm that are done the Right Way, but if we can just drop it altogether that would certainly be easier.
Attachments
Issue Links
Activity
Chris, I don't think there is a huge problem if patches that have LU-1199 land in 2.6 (assuming that they do not need to be updated between now and when they land), since we can just mark this ticket with a fix version for 2.6.0 as well. What is trying to be avoided is releasing 2.5.0 with open tickets. Conversely, we don't want to move tickets with landed patches out from fix version 2.5.0 since that makes it harder to track patches/bugs that were fixed in 2.5.0.
I think cloning the ticket and linking it as a duplicate ensures both that tickets landed in either release can be tracked properly, and doesn't require changing all of the patches in flight.
I meant to suggest that we undo the cloned tickets and restore LU-1199 and its subtasks to active duty.
I'm not a fan of the whole ticket cloning thing that happened here. The LU-1199 is well known as the "random little build system cleanup" ticket number for minor patches. I assume this was done because LU-1199 has Fix Version/s listed for 2.5.0, and you want to close as many 2.5.0 tickets as possible.
I would recommend instead that we remove all Fix Version/s from LU-1199, and just let the sub-tasks be more targeted to specific versions and closed as they are complete.
We also have the problem that in-flight patches now have the wrong jira ticket numbers. For instance, Minh's "Make lustre-iokit a subpackage of lustre" patch. I'd rather not blow away all of the reviews and testing on that patch just to correct the ticket number...but we really should make the ticket number in the patch match the ticket that it goes with.
Remaining subtasks and patches for this ticket will be tracked under LU-3953 for 2.6. Any additional patches for this work should be tracked under the new ticket.
Minh, could you please start to refresh the lustre.spec.in cleanup patches starting with http://review.whamcloud.com/5486.
Yes, I had already started that split, since this ticket has kind of evolved into a build system catch-all, and I don't want to lose sight of my main goal.
We track all of the bugs that have patches landed in each release so that it is possible to query in Jira. If you want to split this bug into smaller sub-tasks it would be easier to track what parts landed in each release.
While some early patches landed for 2.4.0, the main thrust of this ticket was not addressed for Lustre 2.4.0. I'm not sure that adding a Fix Version of Lustre 2.4.0 is appropriate.
I have broken some of the remaining work out into subtasks.
I very much want to see LU-3462 completed for Lustre 2.5.0. I also think it is reasonable to get LU-3464 and LU-3466 done the same time frame.
I would also like to see LU-3463 begun as soon as possible, but I am much less optimistic that it can be completed in time for 2.5.0. Perhaps it can stand next to the current buildfarm and be in an experimental usage mode at that time.
I see little value in having the tracking ticket marked fixed in a release where the work is clearly unfinished. Further, what bothers me more than duplicating the tracking ticket is the cloning of all of the subtasks, which serves no clear purpose. The subtask cloning just makes everything very hard to understand in jira.
The subtask is usually much more simple and clean. There is no good reason to have it appear under two different tracking tickets. We have two main subtask states: finished and unfinished.
Finished subtasks should not be cloned forward to the new tracker. There is no value there. They were finished and closed under the old tracker, no need to clone.
Then there are the unfinished tasks, there is no reason for them to remain connected to the old tracker since they were not completed there. There is very little value (none that I can see) in closing the unfinished ticket under the old tracker and having an new clone ticket opened under the new tracker.
Ultimately I don't really understand the desire to close the tracking ticket against a particular Fix Version when the work is incomplete. I would argue that simply leaving the Fix Version field empty for tracking tickets that are long lived makes more sense.
But if I can't get you folks to agree to that, hopefully you can change your procedures to handle the subtasks more sanely next time.