[LU-11200] Centos 8 arm64 server support

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.14.0
• Affects Version/s: Lustre 2.12.0
• Assignee: James A Simmons
• Reporter: Oleg Drokin

Description

I am trying to bring up arm64 support for centos7 and it turns out centos 7 / rhel 7 for arm64 uses kernel 4.14 that we do not have any ldiskfs patches for.

I checked and the closest match is ubuntu18 4.15 kernel - the ldiskfs series has only a single trivial reject in ext4-data-in-dirent.patch

We also need a way to select it in configure that's a bit more complicated since I don't know how to do it easily yet.

Then there are compile errors:

/home/green/git/lustre-release/lustre/ptlrpc/../../lustre/ldlm/ldlm_lockd.c:135:74: error: macro "DEFINE_TIMER" requires 4 arguments, but only 2 given
 static CFS_DEFINE_TIMER(waiting_locks_timer, waiting_locks_callback, 0, 0);   
                                                                          ^

Issue Links

• is blocking: LU-12323 save_stack_trace_tsk is not exported on arm architectures
• is related to: LU-11224 T10PI assume several kernel features are enabled.
• is related to: LU-11832 ARM servers crashing on MDS startup
• is related to: LU-9793 sanity test 244 fail
• is related to: LU-11878 sanity test 103b: OOM because of too many bash processes: page allocation stalls for 18420ms
• is related to: LU-11246 New lustre e2fsprogs 1.44 issues
• is related to: LU-11440 Make e2fsprogs-1.44.3-wc1 release
• is related to: LU-12137 update client to use iterate_shared
• is related to: LU-10300 Can the Lustre 2.10.x clients support 64K kernel page?
• is related to: LU-11527 sanity test_270a failed with O_DIRECT on ARM
• is related to: LU-6387 Add Power8 support to Lustre
• is related to: LU-6766 add support for arm64

Comments

[Andreas Dilger]

The last patch for this ticket was landed in 2.14, and RHEL8 clients are working for all arches.

[Baptiste Gerondeau (Inactive)]

Thanks a lot !
Will take a look at it this week and report back (here I guess ? Let me know if I should open an issue/if there is a more "à propos" issue elsewhere)

[James A Simmons]

Baptiste I updated https://review.whamcloud.com/#/c/34714 This should make RHLE8 ARM fully functional.

[Baptiste Gerondeau (Inactive)]

I have tested out the latest lustre-release master on RHEL8 ARM64 VM and can confirm that it builds installs, runs and passes (most of) the sanity test suite with LDISKFS (in all-on-one-node configuration at the moment) !

Here are the results : results-ldiskfs-rhel8-2507.yml !
Will investigate further tomorrow, try to isolate the FAILS.

[Baptiste Gerondeau (Inactive)]

I have tested your patches against Fedora 30's 5.1.1 kernel, and the patches do not apply to the ext4.
I have also tested CentOS's 4.14.43-201 and the ialloc.c differs non trivially from what "rhel7.6alt/ext4-corrupted-inode-block-bitmaps-handling-patches.patch" expects.

I'd advise to use the upstream CentOS 7.6alt kernel : here which is the one you get doing a netinstall by default (if you don't run yum update afterwards).
The next to latest one is this one : here
For the latest one, kernel-alt-4.14.0-115.8.1.el7a, I can't seem to find the src rpm but here is the changelog : it doesn't seem to include any changes to EXT4 (only XFS).
I'll try to get a diff of 7.1 and 8.1's ext4 to make sure.

Concerning 7.6 vs RHEL8, I'm happy to test both.
Since the timeline to CentOS 8 is not clear, 7.6alt is still (the most) relevant (and since I haven't tested CentOS 8, there is no assurance it is as stable as 7.6alt yet).

[James A Simmons]

I have resolved the ldiskfs locking issues that ARM testers have reported. Should be in a good position to finish this work off.

[James A Simmons]

We are much closer than you think. The challenge has been getting people to test the needed changes. Now that I have worked out most of the issues for LU-11893 I can focus on finishing this. I talked to Shaun from Cray and he is willing to work with me to finishing this work up.

[Peter Jones]

I am working to the assumption that it will take a little while to get Arm server support working so focusing on something with a longer shelf life would be best

[James A Simmons]

Are you suggesting we drop RHEL7 alt support?  Why that seems like a good idea people might be stuck with what their vendor supports  I think their might be a delay for some time before RHEL8 is picked up ARM vendors. For ORNL we will be getting new ARM servers in soon and one set will be RHEL7 alt and the second set will be RHEL8. Thankfully other vendors haven't made the mistake of rolling their own special ARM only kernels.

[Peter Jones]

Talking to Marvell at ISC this week and they seemed to think that focusing on RHEL8 should be the first priority in this arena.