[LU-10997] Ubuntu 18 support

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0
• Affects Version/s: Lustre 2.12.0
• Environment: Ubuntu 18
• Assignee: James A Simmons
• Reporter: James A Simmons

Description

The next LTS release of Ubuntu is now available. It includes ZFS by default and runs a 4.15 kernel. This ticket will track all the work needed to support this platform.

Issue Links

• is related to: LU-11176 Ubuntu package Issue
• is related to: LU-11635 GSS build for client-only
• is related to: LU-12474 racer can't be found on Ubuntu 18.04
• is related to: LU-10328 sanity-hsm test_13: not enough identical files (87 != 100)
• is related to: LU-10334 Ubuntu1604 client sanity-103a: FAIL: ACL misc test failed
• is related to: LU-10365 sanity test 400a fails with 'client api broken'
• is related to: LU-10366 sanity test 410 fails with 'no inode match'
• is related to: LU-10560 Fixes for 4.14 kernel
• is related to: LU-10575 No mdsrate on Ubuntu client
• is related to: LU-10805 fixes for 4.15 kernel
• is related to: LU-11541 DKMS building the lustre client modules with Mellanox OFED is broken on Debian/Ubuntu
• is related to: LU-12229 ost-pools test 1j fails with 'test_1j returned 1'
• is related to: LU-12282 Pass IB_OPTIONS to debian rules doesn't work
• is related to: LU-13019 sanity test_60g: usleep: command not found
• is related to: LU-13740 Ubuntu 20.04 LTS release
• is related to: LU-11034 update changelog kernel version for ubuntu1804
• is related to: LU-11244 'make debs' mistakenly ignores IB options
• is related to: LU-12137 update client to use iterate_shared
• is related to: LU-10335 Ubuntu1604 client sanity-130a: FAIL: filefrag -ves core dumped
• is related to: LU-10947 conf-sanity test_116: mkfs: failed to execute mkfs.xfs: No such file or directory
• is related to: LU-11575 make debs fail on debian without systemd enabled
• is related to: LU-11246 New lustre e2fsprogs 1.44 issues
• is related to: LU-11071 Support for Lustre Servers on Ubuntu 18.04 LTS
• is related to: LU-9087 Add support for DKMS debs
• is related to: LU-12019 Recognize Debian Kernel in autoconf and unify kernel module package directory

Comments

[James A Simmons]

Now that we are moving to Ubuntu 20 LTS we can close this and move the remaining tickets to that work to resolve.

[James A Simmons]

Last patch  https://review.whamcloud.com/#/c/33492 for complete Ubuntu18 client support

[James A Simmons]

Resolved all the major Ubuntu issues!! Just took many patches

[James A Simmons]

Minh Sebastien found the staging module usage error but now we see the following when building:
checking whether to use Compat RDMA... /usr/bin/ofed_info*10:40:52* xargs: rpm: No such file or directory*10:40:52* configure: error: 10:40:52 You seem to have an OFED installed but have not installed it's devel package.10:40:52 If you still want to build Lustre for your OFED I/B stack, you need to install its devel headers RPM.10:40:52 Instead, if you want to build Lustre for your kernel's built-in I/B stack rather than your installed OFED stack, either remove the OFED package(s) or use -with-o2ib=no.10:40:52 10:40:52 dh_testdir*10:40:52* /usr/bin/make -C . -j 4*10:40:52* make[2]: Entering directory '/var/lib/jenkins/workspace/lustre
 

[James A Simmons]

I noticed the ARM64 is constantly crashing in the recent testing. Its the same reason as Ubuntu18 is now broken :

/usr/src/lustre-2.11.56/lustre/osd-ldiskfs/osd_io.c: In function ‘osd_bio_integrity_compare’:

/usr/src/lustre-2.11.56/lustre/osd-ldiskfs/osd_io.c:345:53: error: ‘struct bio’ has no member named ‘bi_bdev’; did you mean ‘bi_iter’?

  struct blk_integrity bi = bdev_get_integrity(bio->*bi_bdev);

                                                     ^~~~~~~

                                                     bi_iter

scripts/Makefile.build:332: recipe for target '/usr/src/lustre-2.11.56/lustre/osd-ldiskfs/osd_io.o' failed

make[6]: *** [/usr/src/lustre-2.11.56/lustre/osd-ldiskfs/osd_io.o] Error 1

[James A Simmons]

Their is something really wrong with the test setup with Ubuntu. Anything I push fails with:

Trying to load: pxelinux.cfg/1545a47a-0617-0b4b-bafe-af4b2b9c27ce Trying to load: pxelinux.cfg/01-52-54-00-70-0d-c7 Booting from local disk... Could not boot image: Error 0x7f8d81ff (http://ipxe.org/7f8d81ff)

 

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/32304/
Subject: LU-10997 build: add files to .gitignore
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 734d944d98f4cbcaeebc462e979624e8bdfb7711

[James A Simmons]

I see what went wrong. I had some newer ZFS sources in /usr/src which don't match what Ubuntu 18 ships with. So I moved it out of the way. Looking to see where the zfs source are I found them in /usr/src/linux-source-4.15.0/{spl/zfs}. So they are looked in the kernel source itself.  We will have to update the source paths for the Ubuntu case.

[Gerrit Updater]

James Simmons ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32304
Subject: LU-10997 build: add files to .gitignore
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 07056f71248838e678aba5ce8bae2bfcda941343

[James A Simmons]

For my testing I grabbed the master-next branch and rebased https://review.whamcloud.com/#/c/31790

In my initial testing using the default ZFS (0.7.5) that comes with Ubuntu I see the following error on startup.

[312807.501231] Lustre: DEBUG MARKER: /usr/src/lustre-2.11.51/lustre/tests/../utils/.libs/lctl get_param -n version 2>/dev/null || /usr/src/lustre-2.11.51/lustre/tests/../utils/.libs/lctl lustre_build_version 2>/dev/null || /usr/src/lustre-2.11.51/lustre/tests/../utils/.libs/lctl --version 2>
[312807.980495] osd_zfs: Unknown symbol dmu_objset_projectquota_enabled (err 0)
[312807.980574] osd_zfs: Unknown symbol dmu_assign_arcbuf_by_dbuf (err 0)

So looking at the zfs code  ZFS_PROJINHERIT is defined but dmu_objset_projectquota_enabled() is missing. Also I can't seem to find where dmu_assign_archbuf_by_dbuf is being used.