[LU-6387] Add Power8 support to Lustre

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.8.0
• Fix Version/s: Lustre 2.12.0
• Type: New Feature
• Priority: Major
• Reporter: James A Simmons
• Assignee: James A Simmons
• Resolution: Fixed
• Labels: patch

Environment

IBM Power8 hardware. Currently Ubuntu and later RHEL7.1 support will be added to this platform.

Description

Currently working with Power8 client nodes running Ubuntu with a 3.13 kernel. Later the nodes will be moved to RHEL7.1 as support improves.
Here the work will be pushed to be able to build and run lustre in such an
environment.

Issue Links

• is related to: LU-10157 LNET_MAX_IOV hard coded to 256
• is related to: LU-8700 dkms fails to build lustre on Power8 ...
• is related to: LU-8485 workqueue overflows with mlx5 on powe...
• is related to: LU-8693 ko2iblnd recieving IB_WC_MW_BIND_ERR ...
• is related to: LU-10752 Lustre rpm build issues due to improp...
• is related to: LU-11453 sanity test 184a: Basic layout swap p...
• is related to: LU-6284 FLD read is not swabbed correctly
• is related to: LU-8117 lustre-ppc fails to build on ppc64 el7
• is related to: LU-8776 fix weird inline definitions
• is related to: LU-11246 New lustre e2fsprogs 1.44 issues
• is related to: LU-11278 LNet failures on Power8
• is related to: LU-7321 LustreError: 61814:0:(ldlm_lockd.c:69...
• is related to: LU-8567 mdc_reint.c:57:mdc_reint()) error in ...
• is related to: LU-12419 ppc64le: "LNetError: RDMA has too man...
• is related to: LU-11200 Centos 8 arm64 server support
• is related to: LU-11440 Make e2fsprogs-1.44.3-wc1 release
• is related to: LU-10300 Can the Lustre 2.10.x clients support...

Comments

[Gerrit Updater]

James Simmons ([e-mail]) uploaded a new patch: http://review.whamcloud.com/14115
Subject: LU-6387 build: add support for power8
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8dd21c5465f538e76681e20e1c4fd72c29a8aaf0

[Peter Jones]

Jian

Could you please take care of this patch?

Thanks

Peter

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/14115/
Subject: LU-6387 build: add support for power8
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: cb1063a43f3cfee988f801fb791ca4b3a9a1bb76

[Jian Yu]

Patch landed to master branch for Lustre 2.8.0.

[James A Simmons]

Please don't close this ticket yet. Soon we will be moving over to RHEL7.1 so more patches might be needed.

[Gerrit Updater]

James Simmons ([e-mail]) uploaded a new patch: http://review.whamcloud.com/14544
Subject: LU-6387 build: lustre-dev debain package missing shared libraries
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ed6cde3e9e26c616175c5c9ca8f92ef05847b76d

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/14544/
Subject: LU-6387 build: lustre-dev debain package missing shared libraries
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 8ca3a752fc428d35f6ce4cf035dafdfc3d25d4e7

[James A Simmons]

Since this such a new platform it looks like we will be migrating to Ubuntu 15 which uses a patches 3.19 kernel. Except patches soon to support newer kernels.

[Gerrit Updater]

James Simmons ([e-mail]) uploaded a new patch: http://review.whamcloud.com/22941
Subject: LU-6387 tests: fix lp_utils build issues on Power8
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: feda7c747a674eb0a89b1dd0cd3561b9e802532a

[James A Simmons]

Besides the patch I pushed two other issues showed up in this latest port to Power8. The next bug deals with libcfs byteorder which breaks the build. I think we can backport the libcfs byteorder removal to handle that.

The remaining problem I have never seen before. For the test application cascading_rw.c I was seeing missing build ID errors. So the way I worked around this problem was adding the following to lustre.spec.in.

%undefine _missing_build_ids_terminate_build

This enables rpms again but I don't know if that is the correct solution.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/22941/
Subject: LU-6387 tests: fix lp_utils build issues on Power8
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 0d68cfcf18f8f2118d5115fe3766d72c0630bb4d

[James A Simmons]

This also applies to ARM platforms. So the patches you need to make this work for 2.11 out of the box are:

https://review.whamcloud.com/#/c/31485 - So you can build rpms

https://review.whamcloud.com/#/c/31559 - To handle the page size difference

With these I can now run Power/ARM with no problems.

[James A Simmons]

All the last pieces have landed to the lustre community release to support both Power8 and ARM out of the box.

[James A Simmons]

Posted the latest master-lustre-testing e2fsprogs built on Power8.

[James A Simmons]

Here are all the test that fail on Power8 with ZFS server backend.

sanity: FAIL: test_43A execute /lustre/lustre/d43A.sanity/f43A.sanity succeeded

sanity: FAIL: test_56j '/usr/bin/lfs find -type d /lustre/lustre/d56g.sanity' wrong: found 3, expected 4

sanity: FAIL: test_56o lfs find -mtime +0 /lustre/lustre/d56o.sanity: found 0 expect 4

sanity: FAIL: test_56p '/usr/bin/lfs find -uid 2004 /lustre/lustre/d56p.sanity' wrong: found 0, expected 3

sanity: FAIL: test_56q '/usr/bin/lfs find -gid 2647 /lustre/lustre/d56q.sanity' wrong: found 0, expected 3

sanity: FAIL: test_56r '/usr/bin/lfs find -size 5 -type f /lustre/lustre/d56r.sanity' wrong: found 0, expected 1

sanity: FAIL: test_56t '/usr/bin/lfs find -S 8M /lustre/lustre/d56t.sanity' wrong: found 0, expected 3

sanity: FAIL: test_56u '/usr/bin/lfs find -stripe-index 0 -type f /lustre/lustre/d56u.sanity' wrong: found 0, expected 12

sanity: FAIL: test_56wb file was not migrated to pool testpool

sanity: FAIL: test_56y search raid0: found 0 files != 2

sanity: FAIL: test_56ab >16M size files 0 isn't 3 as expected

sanity: FAIL: test_56ba lfs find -E 1M found 0 != 10 files

sanity: FAIL: test_56ca /usr/bin/lfs find --mirror-count 3 --type f /lustre/lustre/d56ca.sanity: 0 != 10 files

sanity: FAIL: test_77g write error: rc=1

sanity: FAIL: test_78 rdwr failed

sanity: FAIL: test_81a write should success, but failed for 28

sanity: FAIL: test_82 test_82 failed with 61

sanity: FAIL: test_103a permissions failed

sanity: FAIL: test_133e Bad write_bytes sum, expected 1376256, got 1409024

sanity: FAIL: test_133f proc file read failed

sanity: FAIL: test_155e dd of=/tmp/f155e.sanity bs=0 count=1k failed

sanity: FAIL: test_155f dd of=/tmp/f155f.sanity bs=0 count=1k failed

sanity: FAIL: test_155g dd of=/tmp/f155g.sanity bs=0 count=1k failed

sanity: FAIL: test_155h dd of=/tmp/f155h.sanity bs=0 count=1k failed

sanity: FAIL: test_241b test_241b failed with 1

sanity: FAIL: test_243 A group lock test failed

sanity: FAIL: test_255c Ladvise test 12, bad lock count, returned  1, actual 0

sanity: FAIL: test_270a file data is different

sanity: FAIL: test_270e lfs find -L: found 0, expected 20

sanity: FAIL: test_270e lfs find -L: found 0, expected 20

sanity: FAIL: test_315 read is not accounted ()

Since both ARM and Power8 use the same kernel version I expect the same failures.