https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-3956 Lustre <p>lbuild's nonstandard build process is making improvements in lustre's build and packaging systems far more difficult than it should be. We really need to move beyond lbuild's non-standard approach.</p> <p>To support proper build procedures for rpm-based distributions, we need a build farm that is capable of building rpms the way that stnards distros like Fedora do it. One approach would be to use <a href="http://fedoraproject.org/wiki/Projects/Mock" class="external-link" target="_blank" rel="nofollow noopener">mock</a>. Another approach would be to boot a new VM instances for every build on every single version of each Linux distribution. The VM instances would be launched from many known clean, basic images.</p> <p>Each piece of software that we want to build should have a .src.rpm. That .src.rpm should have all proper dependencies expressed to allow it to be built under mock and other similar processes.</p> <p><del>We'll need something that sits on top of mock to coordinate the build of dependant packages. At LLNL we use a home-grown script, that honestly has probably exceeded its lifetime.</del></p> <p><del>I would suggest that we again evaluate <a href="http://fedoraproject.org/wiki/Koji" class="external-link" target="_blank" rel="nofollow noopener">Koji</a>, which is Fedora's solution. I haven't used it personally, so I don't know how suitable Koji would be.</del></p> LU-3956

Eliminate lbuild's nonstandard build process

Task Minor In Progress Unresolved Minh Diep Christopher Morrone Mon, 16 Sep 2013 20:40:46 +0000 Wed, 20 Jul 2016 18:26:26 +0000 Lustre 2.5.0 0 5 <p>I am starting this conversion about using Fedora mock to build lustre. First off, I tried the mock command pulling lustre-release git. However, since git repo has lustre.spec.in, mock doesn't understand that and expecting lustre.spec. Any idea?</p> <p>I recommend building the .src.rpm, and then feeding that to mock.</p> <p>Perhaps I should give a bit more of an explanation so it is clear what is going on.</p> <p>By design, Lustre cannot be built from its state in the git repository without first taking a prerequisite action. The build system needs to be bootstrapped into place before any configuration, compilation, and/or packaging can take place.</p> <p>Autotools (autoconf, automake, libtool) form the basis of Lustre's build system. The bootstrapping step is a requirement for any system built on autotools. When it comes to storing your source code in a source repository, there are two schools of thought when it comes to what to do with the products of the bootstrapping process: refresh and commit to the source repository the bootstrap products every time a build system file is modified, or never commit the products and just bootstrap before building. There are advantages and disadvantages to both approaches. Lustre chose the latter; we do not check the bootstrap products into the repository.</p> <p>So when we go to build lustre, the first step is always to bootstrap the build system. We named the bootstrapping script autogen.sh, which is a commonly chosen name. From the base of the source tree, run:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>sh autogen.sh</pre> </div></div> <p>Once autogen.sh is run, it is now possible to configure the build for the specific target platform (by running "configure" and choosing any desired configuration options), and then compile Lustre. That is what a developers commonly do during development. But when we are planning to package Lustre for distribution, we want to take a slightly different approach for the second step.</p> <p>After bootstrapping, we want to generate the canonical full tarball of the source code, ready to be used for further distro packaging, or for end users to untar and compile directly.</p> <p>To make the canonical tarball, we employ the "make dist" target. That target is created by autotools. The Makefiles themselves are generated by autotools as well. So even though we are not ready to configure the build tree for a specific target environment, we need to run the configure script to create the Makefiles that will allow us to create the tarball.</p> <p>It is very important at this step to <em>not</em> make any decisions during the configuration step at this point that would in change the contents of the tarball. That tarball must include the full sources. (The decisions about kernel choice, which OSD, etc. will be made later when the tarball is unpacked and configure is run again to actually test the environment and make decisions about how the sources need to be built.) Since we are not making decisions about the environment yet, we can bypass most of the normal checks that we do when we run <em>configure</em>. In lustre we do that like so:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>./configure --enable-dist</pre> </div></div> <p>It is a struggle to keep the build system clean, so there are probably still some checks in there that are unnecessary, but it should be much better now on master than it was several years ago.</p> <p>And now, we can finally generate the canonical tarball:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>make dist</pre> </div></div> <p>That tarball now contains the .spec files as well. Before bootstrapping and configuration steps, the files were .spec.in as you noted. We employ autotools to fill in a very few number of variables in the spec files. They are: VERSION, DOWNSTREAM_RELEASE, and PACKAGE. All other use of autotools variables in the spec files is forbidden, for reasons I won't go into.</p> <p>So now you have the canonical tarball. Ideally this one tarball would be the input into an entire build system. Different OS build environments would all use that one-time generated source tarball, and not start over from the git repo for each.</p> <p>For rpm-based systems, there is no longer just one .spec file. That is worth keeping in mind. I would not be surprised if we pushed that model further. We might have different spec files for RHEL-like systems and SUSE-like systems, for instance. Already we have a separate spec file for DKMS builds of lustre.</p> <p>Hopefully that provides some illumination on why the spec files have the ".in" extension straight from the git repository.</p> <p>Thanks Chris, that is the most comprehensive explanation I have seen.</p> <p>FYI, I have more experience with koji now, because we are using it to build the TOSS distro here. It would not really be appropriate for building just lustre.</p> <p>I changed the summary and description for this ticket to make it a little more high level. The main goal is really to eliminate the non-standard way that lbuild currently builds things. The specific solution is less important to me.</p> <p>Another approach that may or may not employ mock is to have every build on every version of each distribution start with a cleanly started VM instance. <a href="http://buildbot.net/" class="external-link" target="_blank" rel="nofollow noopener">buildbot</a> makes it pretty easy to generate new VM instances from known clean images on the fly as needed. If we went the mock-less approach, we would still need to keep in mind that we want to have a process that ensures that our .src.rpm remains functional.</p> <p>Perhaps we need to understand more about the build standard; and what so 'non-standard' way that lbuild is doing.<br/> Lbuild is just a script to build lustre ecosystem: zfs, ofed, patch the kernel, and lustre. <br/> Lbuild can build Centos/Rehat/Suse/Ubuntu and can also save the built kernel so the next patch don't have to rebuild kernel.<br/> Lbuild is using rpmbuild which is not so 'non-standard' IMHO.</p> <p>We will need a script to build lustre regardless it's called lbuild or not. Perhaps we need to see what we can improve the script rather declare to eliminate it.</p> <p>In our lab, we are using Jenkins to build not only lustre but many other tools (rpm and non rpm). Jenkins is flexible enough to allow us to achieve that. we have talked about using chroot or generate new VM for every build. However that won't necessary be an improvement and might introduce more overhead to build times</p> <p>However, if you have specific change to lbuild (or any new name), we are please to see a patch <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/> </p> <p>First, read <a href="https://jira.hpdd.intel.com/browse/LU-5614?focusedCommentId=135996&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-135996" class="external-link" target="_blank" rel="nofollow noopener">Stephen Champion's comment</a> in <a href="https://jira.whamcloud.com/browse/LU-5614" title="use %kernel_module_package for weak-updates" class="issue-link" data-issue-key="LU-5614"><del>LU-5614</del></a>. He is exactly right.</p> <p>Everything lbuild does <em>before</em> rpmbuild is non-standard. Here is the most egregious thing that lbuild does:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre> rpm2cpio $rpm | cpio -id </pre> </div></div> <p>It repeatedly sidesteps all standard packing practices by manually tearing apart the packages and putting their contents in random non-standard locations. lbuild's foundations are unsound. We can't fix that with simple patches; it needs a complete overhaul.</p> <p>You can't reasonably address the requirements that Stephen rightly states with a system that repeatedly sidesteps the very packing systems it is targeting.</p> Related LU-5614 LU-3463 Development End date Fri, 11 Dec 2015 20:40:46 +0000 Rank 1|hzw2qn: Rank (Obsolete) 10526 Start date Mon, 16 Sep 2013 20:40:46 +0000