https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-9564 Lustre <p>Currently, only SuSE or RedHat machines can be used as Lustre servers. Ubuntu can only be used as a Client.</p> <p>Since Lustre has recently started supporting SLES12 with Kernel 4.4.0 – the very same version used by Ubuntu 16.04 (and 14.04. via HWE Kernels), we had the idea of porting Lustre over (since our future usage of Lustre would greatly benefit from that).</p> <p>We made good progress in that respect and have adjusted the Kernel-Patches for "ldiskfs" to also work for Ubuntu's flavour of Kernel 4.4.0. Additionally, we have extended the Debian build system provided by Lustre, to be able to create both client and server tools and modules.</p> <p>You can find the patches against Lustre 2.9.57 attached to this ticket. The kernel patches target version 4.4.0-45.66.</p> <p>The compilation and creation of the packages works well and produces the needed modules and tools. Both debian packages install cleanly and the modules (lnet, ldiskfs, lustre) load correctly.</p> <p>Unfortunately, once we come to the creation of the Lustre file system, we run into a curious error:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>root@musxbeo050:~# mkfs.lustre --mgs --mdt --fsname=lustre --backfstype=ldiskfs --index=0 /dev/sda7 mkfs.lustre FATAL: unhandled/unloaded fs type 1 'ldiskfs' mkfs.lustre FATAL: unable to prepare backend (22) mkfs.lustre: exiting with 22 (Invalid argument) </pre> </div></div> <p>This is despite the fact, that "ldiskfs" is properly registered with the kernel:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>root@musxbeo050:~# uname -a Linux musxbeo050 4.4.0-45-generic #66~14.04.1lustre SMP Mon May 8 18:23:05 CEST 2017 x86_64 x86_64 x86_64 GNU/Linux root@musxbeo050:~# grep "ldiskfs\|lustre" /proc/filesystems ldiskfs lustre </pre> </div></div> <p>The only unusual message in the kernel log is a complaint by the "ldiskfs" module, that it cant' register itself under the "ext3" alias.</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>May 26 14:16:11 musxbeo050 kernel: [159891.600363] LDISKFS-fs: Unable to register as ext3 (-16) </pre> </div></div> <p>So far, we have not yet tested the ZFS backend, but since no kernel changes are needed for that one at all, we don't think that it will be a great issue, once this one is solved.</p> <p>Thanks!</p> Ubuntu 14.04.5 with Backport Kernel 4.4.0 from Ubuntu 16.04 LU-9564

Support for Lustre Servers on Ubuntu 14.04/16.04 Kernel 4.4.0

New Feature Minor Resolved Fixed Bob Glossman Martin Schröder ubuntu Fri, 26 May 2017 15:08:07 +0000 Sun, 23 Dec 2018 18:51:28 +0000 Tue, 6 Feb 2018 05:02:48 +0000 Lustre 2.9.0 Lustre 2.11.0 1 17 <p>Please submit patches to Gerrit, so that they can be reviewed. </p> <p>The ldiskfs error appears to be related to the userspace tools (mount_utils_ldiskfs) dynamically loaded, rather than kernel issues. </p> <p>Bob/Others</p> <p>Any advise to give here?</p> <p>Peter</p> <p>Also, given the ZFS support in Ubuntu 16.04, I would think that would be more interesting than ldiskfs - <a href="http://open-zfs.org/wiki/Distributions#Ubuntu" class="external-link" target="_blank" rel="nofollow noopener">http://open-zfs.org/wiki/Distributions#Ubuntu</a></p> <p>I suspect you are missing any lustre capable version of e2fsprogs. Needed for mkfs.lustre of ldiskfs. We have versions available &amp; downloadable for RHEL and SLES, but since we've never done server support for Ubuntu we have none for that.</p> <p>Don't even know if the master-lustre branch of e2fsprogs is capable of building .debs for Ubuntu.</p> <p>I have built the Lustre specific e2fsprogs for Debian Jessie. Probably it can be used for your testing<br/> purpose. It is available here: <a href="http://web-docs.gsi.de/~tstibor/lustre/lustre-builds/jessie/server/2.9/ldiskfs/e2fsprogs-1.42.13.wc5.deb" class="external-link" target="_blank" rel="nofollow noopener">http://web-docs.gsi.de/~tstibor/lustre/lustre-builds/jessie/server/2.9/ldiskfs/e2fsprogs-1.42.13.wc5.deb</a></p> <p>Cheers</p> <p> Thomas</p> <p>Hi everyone.</p> <p>@Peter: Yes, we also want to look at using ZFS at a later point, but for now we want to make sure that we can test both file-systems.</p> <p>@Bob, Thomas: I compiled and deployed the ldiskfs-enabled e2fsprogs for Ubuntu already. I probably should have mentioned that in my original submission. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/tongue.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[-bash-4.3]$ dpkg -l "e2fs*" | grep e2fs un e2fsck-static &lt;none&gt; &lt;none&gt; (no description available) ii e2fslibs:amd64 1.42.13-1-intel amd64 ext2/ext3/ext4 file system libraries ii e2fsprogs 1.42.13-1-intel amd64 ext2/ext3/ext4 file system utilities </pre> </div></div> <p>This is e2fsprogs-1.42.13.wc5, just with a tiny modification to successfully create a DEB on Ubuntu (the file "ext2_types-wrapper.h" is in the wrong location).</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[10:24:55][mhschroe@musxbeo022:/local/mhschroe/e2fsprogs-1.42.13.wc5] [-bash-4.3]$ git log --oneline -3 f77990b Fixed issue with compiling on Ubuntu 00c3728 LU-7867 e2fsprogs: update build version to 1.42.13.wc5 595b511 LU-7867 debugfs: fix check for out-of-bound xattr value </pre> </div></div> <p>Anyway, the same error occurs also with the modified e2fsprogs.</p> <p>@Andreas: I had hoped to upload a working set of changes to Gerrit – especially since my changes to the Automake system and debian/rules will most likely (currently) break your Jenkins config for the Ubuntu Client builds.<br/> But if you think it's useful, I will upload both changesets later today.</p> <p>Martin Schroeder (martin.h.schroeder@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/27323" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27323</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9564" title="Support for Lustre Servers on Ubuntu 14.04/16.04 Kernel 4.4.0" class="issue-link" data-issue-key="LU-9564"><del>LU-9564</del></a> build: Add server-build for Ubuntu with Kernel 4.4.0<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 89cf22fad8717a1ac7c5abf8a2e009e55e334da1</p> <p>Hi everyone.</p> <p>I have now also checked the tool with the ZFS backend, and it has the same depressing result:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>root@musxbeo050:~# dpkg -s "zfsutils" | grep Version Version: 0.6.5.9-1~trusty root@musxbeo050:~# grep "zfs" /proc/filesystems nodev zfs root@musxbeo050:~# modinfo osd_zfs filename: /lib/modules/4.4.0-45-generic/updates/fs/osd_zfs.ko license: GPL version: 2.9.55_78_g1bb015f root@musxbeo050:~# mkfs.lustre --mgs --mdt --fsname=lustre --backfstype=zfs --index=0 /dev/sda7 mkfs.lustre FATAL: unhandled/unloaded fs type 5 'zfs' </pre> </div></div> <p> </p> <p>So the tools are all present and – as far as I can tell – properly compiled and installed.</p> <p>For example, I can create a ZFS pool manually:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>root@musxbeo050:~# zpool create pool-lustre /dev/sda7 root@musxbeo050:~# zpool status pool: pool-lustre state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM pool-lustre ONLINE 0 0 0 sda7 ONLINE 0 0 0 errors: No known data errors </pre> </div></div> <p> </p> <p>By now, I presume that there is actually a problem in the source code of "mkfs.lustre" which prevents it from realizing that, yes, its desired file-systems are indeed available on that Ubuntu Server.</p> <p>I'll try to compile the tool with debugging symbols and then try to check why that error is raised.</p> <p>Good news, everyone.</p> <p>The debugging session helped and showed me that the Debian packages were missing the "osd_&lt;fstype&gt;.so" files to be deployed into /usr/lib.</p> <p>After a quick adjustment of the "debian/*.install" files, a recompile and reinstall, I now get:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>root@musxbeo050:~# mkfs.lustre --mgs --mdt --fsname=lustre --backfstype=ldiskfs --index=0 /dev/sda7 Permanent disk data: Target: lustre:MDT0000 Index: 0 Lustre FS: lustre Mount type: ldiskfs [...] formatting backing filesystem ldiskfs on /dev/sda7 [...] </pre> </div></div> <p><img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p> </p> <p>This means the Ubuntu support should be good now.</p> <p>I've uploaded PatchSet #3 to Gerrit that includes the needed changes for you to review.</p> <p>In the meantime, I'll try seeing how well the system performs.</p> <p> </p> <p>Thanks, everyone for your input!</p> <p>Hi everyone.</p> <p>The change in Gerrit (<a href="https://review.whamcloud.com/#/c/27323/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/27323/</a>) is ready for merge for 12 days now.</p> <p>It has the needed V+1 from Jenkins and Maloo, as well as two CR+1 votes from James and Thomas.</p> <p>What else needs to be done to get this merged? <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>&gt; What else needs to be done to get this merged? </p> <p>The 2.10 code freeze to end <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/27323/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/27323/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9564" title="Support for Lustre Servers on Ubuntu 14.04/16.04 Kernel 4.4.0" class="issue-link" data-issue-key="LU-9564"><del>LU-9564</del></a> build: Add server-build for Ubuntu with Kernel 4.4.0<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 86c3f90d3ab8dbd21dc6fa325aa3a0556fb94035</p> <p>The patch has landed - is that all that is required to complete this task or is further work needed?</p> <p>Hi Peter.</p> <p>The only work that needs to be done is to finish the Wiki dokumentation (it is 95% done):</p> <ul> <li><a href="https://wiki.hpdd.intel.com/pages/viewpage.action?pageId=63968116" class="external-link" target="_blank" rel="nofollow noopener">https://wiki.hpdd.intel.com/pages/viewpage.action?pageId=63968116</a></li> </ul> <p>I should be done with that within the week and will then migrate it to the lustre.org Wiki, too.</p> <p> </p> <p>Beyond that, the only change that needs to be done is to occassionally align the LDISKFS patches with the Ubuntu Kernel releases.</p> <p>Currently it looks like that will only be necessary once every few months, given the rate of changes on the Ubuntu Linux Kernel's 4.4.0 branch.</p> <p>Martin Schroeder (martin.h.schroeder@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/28100" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28100</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9564" title="Support for Lustre Servers on Ubuntu 14.04/16.04 Kernel 4.4.0" class="issue-link" data-issue-key="LU-9564"><del>LU-9564</del></a> build: Added Ubuntu Kernel versions to which_patch and ChangeLog<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: eed1dd8b05304c4b6577c4c0f3274a64e130d051</p> <p>it seems that this patch caused lustre-master build failed</p> <p><a href="https://build.hpdd.intel.com/job/lustre-master/3615/arch=x86_64,build_type=client,distro=ubuntu1604,ib_stack=inkernel/consoleText" class="external-link" target="_blank" rel="nofollow noopener">https://build.hpdd.intel.com/job/lustre-master/3615/arch=x86_64,build_type=client,distro=ubuntu1604,ib_stack=inkernel/consoleText</a></p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">dh_gencontrol: No packages to build. dh_md5sums -p lustre-client-modules-4.4.0-71-<span class="code-keyword">generic</span> dh_builddeb --destdir=/<span class="code-keyword">var</span>/lib/jenkins/workspace/lustre-master/arch/x86_64/build_type/client/distro/ubuntu1604/ib_stack/inkernel/.. -p lustre-client-modules-4.4.0-71-<span class="code-keyword">generic</span> dpkg-deb: error: failed to open <span class="code-keyword">package</span> info file <span class="code-quote">'debian/lustre-client-modules-4.4.0-71-<span class="code-keyword">generic</span>/DEBIAN/control'</span> <span class="code-keyword">for</span> reading: No such file or directory dh_builddeb: dpkg-deb --build debian/lustre-client-modules-4.4.0-71-<span class="code-keyword">generic</span> /<span class="code-keyword">var</span>/lib/jenkins/workspace/lustre-master/arch/x86_64/build_type/client/distro/ubuntu1604/ib_stack/inkernel/.. returned exit code 2 debian/rules:424: recipe <span class="code-keyword">for</span> target <span class="code-quote">'binary-modules'</span> failed make[2]: *** [binary-modules] Error 1 make[2]: Leaving directory <span class="code-quote">'/<span class="code-keyword">var</span>/lib/jenkins/workspace/lustre-master/arch/x86_64/build_type/client/distro/ubuntu1604/ib_stack/inkernel/debian/tmp/modules-deb/usr_src/modules/lustre'</span> /<span class="code-keyword">var</span>/lib/jenkins/workspace/lustre-master/arch/x86_64/build_type/client/distro/ubuntu1604/ib_stack/inkernel/debian/tmp/modules-deb/usr_share_modass/include/common-rules.make:56: recipe <span class="code-keyword">for</span> target <span class="code-quote">'kdist_build'</span> failed make[1]: *** [kdist_build] Error 2 make[1]: Leaving directory <span class="code-quote">'/<span class="code-keyword">var</span>/lib/jenkins/workspace/lustre-master/arch/x86_64/build_type/client/distro/ubuntu1604/ib_stack/inkernel/debian/tmp/modules-deb/usr_src/modules/lustre'</span> tput: No value <span class="code-keyword">for</span> $TERM and no -T specified BUILD FAILED! tput: No value <span class="code-keyword">for</span> $TERM and no -T specified See /<span class="code-keyword">var</span>/lib/jenkins/workspace/lustre-master/arch/x86_64/build_type/client/distro/ubuntu1604/ib_stack/inkernel/debian/tmp/modules-deb/var_cache_modass/lustre.buildlog.4.4.0-71-<span class="code-keyword">generic</span>.1500438566 <span class="code-keyword">for</span> details. autoMakefile:1142: recipe <span class="code-keyword">for</span> target <span class="code-quote">'debs'</span> failed make: *** [debs] Error 7 </pre> </div></div> <p>Hi Minh Diep.</p> <p> </p> <p>Curious that the tests started from Gerrit for the change have not picked that issue up.</p> <p>I will try to replicate it on our Ubuntu servers and get back to y'all. I do admit that in my test cases, I only looked at the "make debs" target in isolation, and never run the "make dist" target as done by this Jenkins test.</p> <p>I'm building my debs against only ZFS but when I run make dist I see:</p> <p>tardir=lustre-2.10.50_68_gb2c8846_dirty &amp;&amp; tar -<del>format=ustar -chf - "$tardir" | GZIP=</del>-best gzip -c &gt;lustre-2.10.50_68_gb2c8846_dirty.tar.gz<br/> tar: lustre-2.10.50_68_gb2c8846_dirty/debian/lustre-source/usr/share/modass/overrides/lustre-source: File removed before we read it<br/> tar: lustre-2.10.50_68_gb2c8846_dirty/debian/tmp/modules-deb/usr_src/modules/lustre/debian/lustre-client-utils.manpages: link name is too long; not dumped</p> <p>This is on Ubuntu 17.</p> <p>Oleg Drokin (oleg.drokin@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/28293" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28293</a><br/> Subject: Revert "<a href="https://jira.whamcloud.com/browse/LU-9564" title="Support for Lustre Servers on Ubuntu 14.04/16.04 Kernel 4.4.0" class="issue-link" data-issue-key="LU-9564"><del>LU-9564</del></a> build: Add server-build for Ubuntu with Kernel 4.4.0"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 94ed9f023bfeadcdf25202d2d0432339b9fd71b0</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/28293/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/28293/</a><br/> Subject: Revert "<a href="https://jira.whamcloud.com/browse/LU-9564" title="Support for Lustre Servers on Ubuntu 14.04/16.04 Kernel 4.4.0" class="issue-link" data-issue-key="LU-9564"><del>LU-9564</del></a> build: Add server-build for Ubuntu with Kernel 4.4.0"<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 2deec088aa4e1153bc00730c2b1b4bf5d40d275a</p> <p>Please resubmit this patch in a form that actually works in our build system (mix in the changelog/which_patch/... update too please)</p> <p>Thanks.</p> <p>Will do that, yes.</p> <p> </p> <p>Of course, this only raises the point that the build system that covers the uploads does not actually cover all the build types that you use.</p> <p>From what I can tell, it covers only the "make debs" path on Ubuntu (which works) – but not the "make dist" call (which does not).</p> <p> </p> <p>Indeed, I never even knew that there was an alternative make rule for building Debs to begin with. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/tongue.png" height="16" width="16" align="absmiddle" alt="" border="0"/> </p> <p> </p> <p><b>So, my question would be:</b> What would it take to have both "make debs" and "make dist" be checked by your Jenkins builds that trigger on Gerrit upload?</p> <p> </p> <p>EDIT: Typo correction – it is "make dist", not "make site".</p> <p>Martin</p> <p>Yes it will be possible to make these checks routine once they are reliably working</p> <p>Peter</p> <p>Hi everyone.</p> <p>I have finally found the time to look at this, and I can't replicate the issue that was reported.</p> <p>I installed two VMs – one with Ubuntu 14.04 and one with 16.04 and ran the DEB creation sucessfully on both of them, while following the build steps used by the HPDD build server, namely in the form of these commands:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>$ sh autogen.sh &amp;&amp; ./configure --enable-dist $ make dist $ rm -rf BUILD $ mkdir BUILD $ cd BUILD $ tar -zxf ../lustre-2.10.53_1_ga949a00.tar.gz $ cd lustre-2.10.53_1_ga949a00 $ ./configure --with-linux=${BUILDPATH}/ubuntu-kernel --disable-server $ make debs </pre> </div></div> <p>The kernel that was used to compile Lustre against was 4.4.0-71 in both cases – the same that was used by the failing run above.</p> <p> </p> <p>Hence, I can only conclude that it was a one-off failure on the build system – or is something specific to the environment / kernel sources used by the HPDD build server.</p> <p>I will shortly upload a new changeset to Gerrit, so that it can be tested on the build-server.</p> <p> </p> <p>Thanks,<br/>     Martin.</p> <p>Martin Schroeder (martin.h.schroeder@intel.com) uploaded a new patch: <a href="https://review.whamcloud.com/29215" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29215</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9564" title="Support for Lustre Servers on Ubuntu 14.04/16.04 Kernel 4.4.0" class="issue-link" data-issue-key="LU-9564"><del>LU-9564</del></a> build: Add server-build for Ubuntu with Kernel 4.4.0<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: a949a00df7ccb18e054f3b9acc6b5ab104dff535</p> <p>Hi everyone.</p> <p>Just as a heads-up. I narrowed the bug down. The code builds perfectly fine on Ubuntu 14, but fails on Ubuntu 16 while assembling the DEB for the client-modules.<br/> It seems they changed some things in the "debhelper" macros that used to work on earlier versions deployed in 14.04 and early 16.04, but fails in the recent updates to 16.06.</p> <p>Fun times. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/biggrin.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>{EDIT}<br/> Found it!</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java">Build-Depends: debhelper (&gt;= 7.0.0), bzip2 </pre> </div></div> <p>The "7.0.0" must not be used. It seems that the current way of building modules (via modules-assistant) confuses newer debhelper variants.</p> <p>Once I reverted this to request version "5.0.0", the build works on 16.04. now, too.</p> <p>Have you tried upping the debhelper version. I found on my Ubuntu17 system it complains about debhelper using protocol 7 instead of 9 which is the supported default.</p> <p>Hi James.</p> <p>Thanks, as I said above (in the edit), it works when requesting an even <em>older</em> version, namely 5.0.0.</p> <p>Since the original code uses 5.0.0 I'll keep it at that version, but I'll also check it with "9.0.0" and update this post, once it's done.</p> <p>{EDIT}<br/> Correction: It seems that it is not the debhelper version, but the presence/absence of the "Build-Profiles" section I had added. Once I removed those, it worked with all versions equally.</p> <p>One never stops learning. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/biggrin.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Hi everyone.</p> <p>The Gerrit change which fixes all outstanding issues has been uploaded 3 weeks ago and has passed all automated tests.</p> <p>I wonder what else is needed to entice users that have CR rights to review and ultimately merge the changes? <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p> </p> <p>Would it help if I got out of the spaceship and pushed it?  <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/wink.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Sorry I saw previous comments but didn't realize you updated the patch. Will look now. I will the proper people to review.</p> <p>the latest kernel version on Ubuntu 16 appears to be 4.10, not 4.4. Is that supported?</p> <p>We officially support the 4.4 kernel version because that is a known quantity and allows us to maintain support for Ubuntu 16 regardless of what kernel version is used the cutting edge version. We do attempt to keep as abreast with the latest kernel versions as possible so most of the time I would expect us to be able to support either option. James will be able to comment authoritatively but I think that we're able to support up to 4.11 and are very close to 4.12 so I would expect 4.10 to be ok.</p> <p>Hi everyone.</p> <p>I can only assent to what Peter said. I chose Kernel 4.4.0, because of several reasons:</p> <ul> <li>It is the Kernel shipped with the original Ubuntu 16.04</li> <li>It is supported via backports on Ubuntu 14.04 (and is default on 14.04.5)</li> <li>It is already supported by the Lustre team for RedHat/SuSE, so I was able to re-use the existing Kernel patches with minimal changes.</li> </ul> <p>In other words: I went the path of least resistance. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/biggrin.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p> </p> <p>From what I have seen when adjusting the patches for Ubuntu's 4.4.0 version, I do not think it would be a great problem to target 4.10 -- <em>eventually.</em><br/> After all, the build scripts already dynamically detect the Kernel version and choose the correct set of patches.</p> <p> </p> <p>But to get this patch merged, I'd vote for sticking with 4.4.0 for a while, until someone with more experience than me regarding Lustre Kernel patches can find the time to port them to 4.10. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Martin you are going to need to rebase the patch : -). It failed to apply when I went to try it. </p> <p>As for newer kernel support we are one patch away from 4.12 kernel support for the client. I have updated the patch, which is <a href="https://review.whamcloud.com/#/c/28511" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/28511</a>. For lustre 2.10 we are one patch away from 4.11 kernel support. This is also assuming GSS keyring is disabled which is stuck at pre-4.6 kernels.</p> <p>Also the latest ZFS + lustre with a 4.12 kernel works fine. Its ldiskfs which I haven't touched for a 4.12 kernel. I haven't tried to port ldiskfs to any newer kernels.</p> <p>Latest patch isn't working for me at all.<br/> 'build debs' fails with errors like the following:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre># Create the module-source tarball. cd debian/lustre-source/usr/src &amp;&amp; tar jcf lustre.tar.bz2 modules rm -rf debian/lustre-source/usr/src/modules dh_install -plustre-source dh_installchangelogs -p lustre-source lustre/ChangeLog dh_installdocs -p lustre-source dh_link -p lustre-source /usr/share/modass/packages/default.sh /usr/share/modass/overrides/lustre-source dh_compress -p lustre-source dh_installdeb -p lustre-source dh_fixperms -p lustre-source dh_gencontrol -p lustre-source dh_md5sums -p lustre-source dh_builddeb -p lustre-source dpkg-deb: building package 'lustre-source' in '../lustre-source_2.10.54-14-g65983ee-1_all.deb'. dh_testdir dh_testroot dh_installdirs -p lustre-server-utils dh_installdocs -p lustre-server-utils dh_installman -p lustre-server-utils dh_install -p lustre-server-utils dh_install: lustre-server-utils missing files: debian/tmp/usr/lib/lustre/*.la dh_install: missing files, aborting debian/rules:239: recipe for target 'binary-lustre-server-utils' failed make[1]: *** [binary-lustre-server-utils] Error 255 make[1]: Leaving directory '/home/bogl/lustre-release' dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2 autoMakefile:1142: recipe for target 'debs' failed make: *** [debs] Error 2 </pre> </div></div> <p>It needs a rebase as well.</p> <p>that fail was a rebase on latest master</p> <p>Okay, I'll rebase and see where the problem might be.</p> <p> </p> <p>I do remember the "missing files" issue, which happened when certain configurations would not build shared libraries, only static ones. But I believed to have fixed all of them. Let me check. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Its a libtool thing. Looking on my ubuntu system I don't see any *.la files installed. I noticed we are installing them.</p> <p>Hi everyone.</p> <p>I have rebased the code, fixed the build-time issues and changed the DEB file creation to allow more than one "lustre - * - modules - &lt;KVERS&gt;" package to be installed simultaneously.</p> <p>I have checked the compilation on both Ubuntu 14.04 and 16.04. and against the most recent Linux Kernel version available for both.</p> <p>If you have the time, please review the Gerrit change under: <a href="https://review.whamcloud.com/#/c/29215" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/29215</a></p> <p>Do note especially the small workaround for "IB_DEVICE_SG_GAPS_REG" and "IB_MR_TYPE_SG_GAPS", that is needed to compile against the Ubuntu Kernels, which do not have those macros. The change to "lnet/klnds/o2iblnd/o2iblnd.c" should be broadly compatible with both older and newer kernels.</p> <p> </p> <p>Thanks!</p> <p>For "IB_MR_TYPE_SG_GAPS" you might want to look at <a href="https://jira.whamcloud.com/browse/LU-10089" title="kiblnd_fmr_pool_map() Failed to map mr 10/11 elements" class="issue-link" data-issue-key="LU-10089"><del>LU-10089</del></a></p> <p>Hi Chris. I've seen that commit (and an earlier one that was already merged) when I encountered the compile-time issue.</p> <p>The problem here is, that neither "IB_DEVICE_SG_GAPS_REG" nor "IB_MR_TYPE_SG_GAPS" are available on the 4.4.0-series Kernels as used by Ubuntu at all. So any code that mentions them outside of a suitable preprocessor guard will fail to compile. In this case, the guard matched, even when the symbols were not present.</p> <p> </p> <p>The reason for their absence appears to be, that there is a general warning against using these symbols:</p> <ul> <li><a href="https://patchwork.kernel.org/patch/9573483/" class="external-link" target="_blank" rel="nofollow noopener">https://patchwork.kernel.org/patch/9573483/</a></li> <li><a href="https://lkml.org/lkml/2017/3/13/206" class="external-link" target="_blank" rel="nofollow noopener">https://lkml.org/lkml/2017/3/13/206</a></li> </ul> <p>So either they removed them, or they never used them on the 4.4.0 Kernel.</p> <p> </p> <p>I changed the code so that Lustre will use them, if the symbols are present. If missing, it will revert to the old pre-patch behaviour.<br/> At least that's what I hope my code does. Checking for symbols via preprocessor macros can be iffy, if what you check turns out not to be a macro in the first place. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/tongue.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p> </p> <p>I would prefer to see those o2iblnd.c changes as a separate patch. That fix isn't specific to server builds. It is needed for any build on current Ubuntu kernel versions, including client builds.</p> <p>Okay, I will extract the o2iblnd.c changes into a separate changeset and then adjust the dependencies of the current one, so that it depends on the other. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>Add reference to blocking subtask for o2iblnd.c changes.</p> <p>@Bob Glossmann: Please review: <a href="https://review.whamcloud.com/#/c/30893/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/30893/</a></p> <p>Okay everyone.</p> <p>Both changes have CR+1/V+1. Now, I guess all that is needed is to find someone with the needed permissions to push the code into the mainline.</p> <p>So... any volunteers on the front of merging the two commits? <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p> </p> <p>Once the code is merged and shown to not cause any problems (like last time), it would also be a good idea to add a Ubuntu Server build to the Jenkins config. The Client build is already there, after all. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/biggrin.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p> </p> <p>Thanks,<br/>     Martin.</p> <p>Martin</p> <p>The commits will be landed by the gatekeeper when he has completed necessary reviews and tests. One of the patches is in the batch being processed ATM - <a href="https://git.hpdd.intel.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master-next" class="external-link" target="_blank" rel="nofollow noopener">https://git.hpdd.intel.com/?p=fs/lustre-release.git;a=shortlog;h=refs/heads/master-next</a> - and the other will likely be in the next batch</p> <p>Peter</p> <p>Oleg Drokin (oleg.drokin@intel.com) merged in patch <a href="https://review.whamcloud.com/29215/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/29215/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-9564" title="Support for Lustre Servers on Ubuntu 14.04/16.04 Kernel 4.4.0" class="issue-link" data-issue-key="LU-9564"><del>LU-9564</del></a> build: Add server-build for Ubuntu with Kernel 4.4.0<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 027a7237b560489099ba490db925db17a554f37d</p> <p>Landed for 2.11</p> Blocker LU-10526 Related LU-11071 LU-10526 Development Epic/Theme ubuntu Rank 1|hzzdnr: Rank (Obsolete) 9223372036854775807