https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9775 Lustre <p>If one is building in a build <span class="error">&#91;ch&#93;</span>root such as <em>mock</em> provides, one may not have the kernel installed which corresponds to <tt>$(uname -r)</tt>.</p> <p>In such a case, also try to look for the <em>kernel-devel</em> in <tt>/usr/src/kernels/</tt> and just build for the latest one.  Ideally there is only one installed in any case.</p> LU-9775

find kernel-devel even if the current kernel is not installed in the build root

Improvement Minor Resolved Fixed Brian Murrell Brian Murrell Fri, 14 Jul 2017 12:40:34 +0000 Fri, 18 Aug 2017 21:21:07 +0000 Mon, 24 Jul 2017 17:46:19 +0000 Lustre 2.10.0 Lustre 2.10.1 Lustre 2.11.0 0 8 <p>Brian J. Murrell (brian.murrell@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/28046" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28046</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9775" title="find kernel-devel even if the current kernel is not installed in the build root" class="issue-link" data-issue-key="LU-9775"><del>LU-9775</del></a> Look for kernel-devel in /usr/src/kernels<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: 1<br/> Commit: 59dd5a89fcde9f5ec48656591a4401e9219028ec</p> <p>/usr/src/kernel is very RHEL specific. Would this break non-RHEL systems? On a system we could also end up with variants like PAE or XEN so it can be even more complex.</p> <blockquote> <p>/usr/src/kernel is very RHEL specific.</p></blockquote> <p>Agreed, which is why I put it at the end of the search list.</p> <blockquote> <p>Would this break non-RHEL systems?</p></blockquote> <p>None that I am aware of. I don't know of any other systems using <tt>/usr/src/kernels/</tt>. If there were, and they were doing it in an incompatible way, of course we could make this search dependent on the distro <tt>configure</tt> is being run on. That just doesn't seem necessary at this point so the complexity seems unnecessary at this point also.</p> <blockquote> <p>On a system we could also end up with variants like PAE or XEN so it can be even more complex.</p></blockquote> <p>I'm not sure this is very likely. <tt>/usr/src/kernels/</tt> is at the end of the search list and so only applies to a system that doesn't actually have a kernel matching <tt>$(uname -r)</tt> installed and as such the <tt>/lib/modules/$(uname -r)/{source,build</tt>} links are missing. – hence the need to keep looking for some kernel source.  I can't imagine a system that this would be true on other than a <em>mock</em>-style (i.e. <em>chroot</em>) build environment and in such an environment, you would only populate it with the <em>kernel-devel</em> of the kernel you are wishing to build for.</p> <p>Maybe there are cases I am not thinking of?</p> <p>If appropriate kernel-devel is installed into <tt>/usr/src/kernels/&lt;kver&gt;/</tt> directory it will always have a correct symbolic link from <tt>/lib/modules/&lt;kver&gt;/build</tt>. So, probably it will be better to just create those links for kernel-devel's without appropriate kernel's. Many scripts use those links instead of direct location.</p> <p> </p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=dmiter" class="user-hover" rel="dmiter">dmiter</a>: Not all build environments provide the ability to "fiddle" with the build <span class="error">&#91;ch&#93;</span>root that the build is being done in.  Probably for good reason.  You "taint" the "clean room" by allowing it to be fiddled with before the build.</p> <p><a href="https://pagure.io/copr/copr" class="external-link" target="_blank" rel="nofollow noopener">Copr</a> is one such type build environment.</p> <p>Brian J. Murrell (brian.murrell@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/28064" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28064</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9775" title="find kernel-devel even if the current kernel is not installed in the build root" class="issue-link" data-issue-key="LU-9775"><del>LU-9775</del></a> Look for kernel-devel in /usr/src/kernels<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 48edcc4bb7a7985d9fdb90e45aa01e2fc3c405bf</p> <p>Don't think this particular patch is good or useful. In the common case where links in /lib/modules are missing and it would go to picking one from /usr/src/kernels, the one it picks is fairly arbitrary. It isn't the case that there is only one most of the time.</p> <p>If you must pick from there it would be better to first look for a match on 'uname -r', first an exact match and then a close match if no exact match.</p> <p>In general I think putting in appropriate symlinks in /lib/modules as Dmitry suggests even if the correct kernel-devel isn't installed would be better.<br/> btw, kernel-devel of pristine, unpatched upstream linux is insufficient for building ldiskfs. In RHEL distros it doesn't include ext4 sources, needed for building ldiskfs. kernel source or debuginfo is needed</p> <p>On a test with symlinks deleted from /lib/modules to see what would happen this patch doesn't even work. I get the following failure:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>./configure --disable-server . . . checking whether to build Lustre client support... yes checking whether mpitests can be built... yes checking whether to build Linux kernel modules... yes (linux-gnu) checking for Linux sources... /usr/src/linux checking for /usr/src/linux... no configure: error: Kernel source /usr/src/linux could not be found. </pre> </div></div> <p>It appears the current logic looks in /usr/src/linux first and if there is no such dir it doesn't even get as far as looking in /usr/src/kernels.</p> <p>There isn't any /usr/src/linux in RHEL/Centos installs</p> <p>I believe the logic to look at /usr/src/linux is there to cater to SLES installs. In SLES /usr/src/linux is a symlink to the linux tree corresponding to the boot kernel.</p> <blockquote> <p>Don't think this particular patch is good or useful.</p></blockquote> <p>Have you ever built Lustre in a "clean-room" (i.e. with <tt>mock</tt>)? That's one case where it's useful.</p> <p>To be contrarian, does adding another element to the list of places searched for kernel source cause any harm?</p> <blockquote> <p>In the common case where links in /lib/modules are missing and it would go to picking one from /usr/src/kernels, the one it picks is fairly arbitrary.</p></blockquote> <p>It's not actually. Even in the case of there being multiple entries in <tt>/usr/src/kernels/</tt> – which I don't think is actually all that common at all for the use-cases that this patch is targeting – but in any case, it will pick the highest versioned kernel, which seems like the least surprising choice to me. If that is not what the user wants, then he can still use <tt>--with-linux</tt> to point at the one he wants, but using the most recent seems like a reasonable default when all other selection mechanisms have been exhausted.</p> <blockquote> <p>It isn't the case that there is only one most of the time.</p></blockquote> <p>Even in environments where all of the other search locations have not found kernel source? I am doubtful that your assertion is true. I suspect environments that this patch is targeting are few and are particular, typically about clean-room isolated build environments.</p> <blockquote> <p>If you must pick from there it would be better to first look for a match on 'uname -r',</p></blockquote> <p>As has been discussed previously in this ticket, if there was a match for <tt>$(uname -r)</tt> then the targeted kernel is what would be installed and booted already and the <em>/lib/modules/$(uname -r)/{build,source}</em> symlinks would be present. This patch is specifically targeting build environments where the booted kernel (the one that responds to <tt>$(uname -r)</tt>) is not the kernel you want to build for in your build chroot.</p> <blockquote> <p>In general I think putting in appropriate symlinks in /lib/modules as Dmitry suggests even if the correct kernel-devel isn't installed would be better.</p></blockquote> <p>In a "clean-room" build system (or BaaS – Builder as a Service) you are usually not allowed to mess with the "clean" room.</p> <blockquote> <p>btw, kernel-devel of pristine, unpatched upstream linux is insufficient for building ldiskfs. In RHEL distros it doesn't include ext4 sources, needed for building ldiskfs. kernel source or debuginfo is needed</p></blockquote> <p>This is not for building patched servers. It's for building the patchless client.</p> <blockquote> <p>On a test with symlinks deleted from /lib/modules to see what would happen this patch doesn't even work. I get the following failure:</p></blockquote> <p>Bah.  Good catch.  That was due to an unfortunate nastiness with <em>m4</em> macros.  I've updated the patch.  Now I get this:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>$ ./configure --disable-server ... checking whether to build Lustre client support... yes checking whether mpitests can be built... no checking whether to build Linux kernel modules... yes (linux-gnu) checking for Linux sources... /usr/src/kernels/3.10.0-514.26.2.el7.x86_64 checking for /usr/src/kernels/3.10.0-514.26.2.el7.x86_64... yes checking for Linux objects... /usr/src/kernels/3.10.0-514.26.2.el7.x86_64 checking for /usr/src/kernels/3.10.0-514.26.2.el7.x86_64/.config... yes checking for /boot/kernel.h... no checking for /var/adm/running-kernel.h... no checking for /usr/src/kernels/3.10.0-514.26.2.el7.x86_64/include/generated/autoconf.h... yes checking for /usr/src/kernels/3.10.0-514.26.2.el7.x86_64/include/linux/version.h... no checking for /usr/src/kernels/3.10.0-514.26.2.el7.x86_64/include/generated/uapi/linux/version.h... yes checking for /usr/src/kernels/3.10.0-514.26.2.el7.x86_64/include/linux/kconfig.h... yes </pre> </div></div> <blockquote> <p>It appears the current logic looks in /usr/src/linux first and if there is no such dir it doesn't even get as far as looking in /usr/src/kernels.</p></blockquote> <p>That's not how the code reads:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"><span class="code-keyword">for</span> DEFAULT_LINUX in /lib/modules/$(uname -r)/{source,build} /usr/src/linux $(find /usr/src/kernels/ -maxdepth 1 -name [0-9]\* | xargs -r ls -d | tail -n 1); <span class="code-keyword">do</span> AS_IF([readlink -q -e $DEFAULT_LINUX &gt;/dev/<span class="code-keyword">null</span>], [<span class="code-keyword">break</span>]) done </pre> </div></div> <p>That says to iterate through the list of <tt>/lib/modules/$(uname -r)/{source,build} /usr/src/linux $(find /usr/src/kernels/ -maxdepth 1 -name <span class="error">&#91;0-9&#93;</span>* | xargs -r ls -d | tail -n 1)</tt> and stop (<tt>break</tt>) when a list item is valid. If <tt>/usr/src/linux</tt> is not valid, whatever <tt>$(find /usr/src/kernels/ -maxdepth 1 -name <span class="error">&#91;0-9&#93;</span>* | xargs -r ls -d | tail -n 1)</tt> evaluates to will be tried next.</p> <p>you said</p> <blockquote> <p>This is not for building patched servers. It's for building the patchless client.</p></blockquote> <p>As I understand it future plans call for building unpatched servers too. Building any server, patched or unpatched, calls for building ldiskfs. Any build of ldiskfs requires ext4 source.</p> <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=bogl" class="user-hover" rel="bogl">bogl</a> Fair enough.  But that's a future concern with a number of possible solutions, none of which have probably been decided on at this point.  So it seems unreasonable to prevent solving today's problems just because of not yet knowing how we will solve tomorrow's problem.</p> <p>Once tomorrow comes, and the solution for tomorrow's problem has been decided on, yes, today's solution might need altering.  But so might so many others.  That is part of developing solutions in lively project.</p> <p>IMHO, when we do go to a patchless server and still need <em>ldiskfs</em> (i.e. because we cannot get ext* upstream to accept the patches that make <em>ext4</em> linto diskfs), it's probably time for <em>ldiskfs</em> to be maintained as a standalone module built as a kmod and/or with DKMS.  In the kmod case, hopefully we could get the module kABI compliant so that it doesn't need to keep being rebuilt for successive RHEL minor releases.</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/28064/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28064/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9775" title="find kernel-devel even if the current kernel is not installed in the build root" class="issue-link" data-issue-key="LU-9775"><del>LU-9775</del></a> Look for kernel-devel in /usr/src/kernels<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 1a8c125e39f1a96c1afcbd5fac88c34e92d63eff</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/28046/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28046/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9775" title="find kernel-devel even if the current kernel is not installed in the build root" class="issue-link" data-issue-key="LU-9775"><del>LU-9775</del></a> Look for kernel-devel in /usr/src/kernels<br/> Project: fs/lustre-release<br/> Branch: b2_10<br/> Current Patch Set: <br/> Commit: ad7e776c076108dfa2b64120cd80c8adb5dafda2</p> <p>removed unrelated gerrit upload message</p> Development Rank 1|hzzgof: Rank (Obsolete) 9223372036854775807