[LU-10300] Can the Lustre 2.10.x clients support 64K kernel page?

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.10.0
• Type: Question/Request
• Priority: Major
• Reporter: ZhangWei
• Assignee: James A Simmons
• Resolution: Cannot Reproduce

Environment

Redhat

Description

We test Lustre in the kernel 4.11.0 which use 64K memory page, and we found that there are no config items in the configure.

Issue Links

• is related to: LU-14970 sanity-flr test_50a: FAIL: Mirrored f...
• is related to: LU-14346 sanity-pcc test 7a hangs in multiop/d...
• is related to: LU-10157 LNET_MAX_IOV hard coded to 256
• is related to: LU-15364 Kernel oops when stripe on Arm64 Serv...
• is related to: LU-7650 ko2iblnd map_on_demand can't negotita...
• is related to: LU-11597 sanityn test 16a failed with direct I/O
• is related to: LU-11785 conf-sanity test_98 fails with 'Buffe...
• is related to: LU-11788 sanity test 104a fails with ‘lfs df f...
• is related to: LU-6387 Add Power8 support to Lustre
• is related to: LU-11200 Centos 8 arm64 server support
• is related to: LU-15293 Add lbuild support for latest Arm64 C...
• is related to: LU-11671 sanity test 45: FAIL: write wasn't ca...
• is related to: LU-12255 sanity-dom test 42e fails with ‘test...
• is related to: LU-4398 mdt_object_open_lock() may not flush ...
• is related to: LU-10073 lnet-selftest test_smoke: lst Error f...
• is related to: LU-11729 ARM: sanity test_810: BAD WRITE CHECK...
• is related to: LU-11595 sanity-dom sanityn test 11: LBUG: (fi...
• is related to: LU-11596 sanity test_42d/test_42e: FAIL: faile...
• is related to: LU-12014 check correct size in ll_dom_finish_o...
• is related to: LU-15722 IO write gets stuck on some sanityn t...
• is related to: LU-11667 sanity test 317: FAIL: Expected Block...
• is related to: LU-11787 sanityn test 71a fails with ‘data is ...
• is related to: LU-12362 kernel warning 'do not call blocking ...
• is related to: LU-12419 ppc64le: "LNetError: RDMA has too man...
• is related to: LU-15223 Improve partial page read/write

Comments

[James A Simmons]

Actually I have tested such a setup on Power8 which is 64K page based. The only issue that came up was when dealing with the ko2iblnd witth map_on_demand enabled. Otherwise it works.

[ZhangWei]

Thanks for your replay, with the map_ondemand enabled, can the Lustre only support 4k kernel memory page?

[Andreas Dilger]

We definitely used to run Lustre with 64KB PAGE_SIZE on IA64 clients, and in theory this would still work but we haven't tested it in a long time.  We never had much success with 64KB PAGE_SIZE on the server, since this caused problems with 4KB PAGE_SIZE clients doing writes to the 64KB PAGE_SIZE server.

[ZhangWei]

Yes, I tested the 64KB PAGE_SIZE server with 4KB PAGE_SIZE Client and 64KB PAGE_SIZE Client, there are some errors both in these two clients.Thanks for help !

[Andreas Dilger]

It probably makes sense to print a build warning in the server code when PAGE_SIZE isn't 4096, just so people are aware that this isn't being tested. If someone starts testing this in the future, the warning can be removed.

[James A Simmons]

Once the RHEL ARM/Power8 server support work is complete we can test 64K pages on the server side.

[James A Simmons]

So I have managed to get Lustre ZFS servers running on Power8 nodes. I got it to mount and then it just locked up with any attempt to use the file system  Well its a start

[James A Simmons]

This time I tested ZFS lustre servers using the ethernet interface and it worked. Currently their is a bug in the RHEL7 alt kernel that I'm using in netlink that shows up when using k2oiblnd for some reason. Their appears to be a fix but requires rebuild the RHEL7 alt kernel. Now to test ldiskfs.

[Andreas Dilger]

James, I'd suggest to move your previous comment into a separate ticket related to 64KB PAGE_SIZE on the server, and leave this one for tracking 64KB PAGE_SIZE on the client. I suspect most people care about client-side support more than server-side, and this will help understand which issues are important to fix for the two different cases. In any case, it doesn't make sense to have multiple independent issues being worked on in the same Jira ticket.

[Gerrit Updater]

"James Simmons <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/45962
Subject: LU-10300 test: does arm testing work
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2036c3298804defa44e0457e65893e91d1da3985

[Andreas Dilger]

The aarch64 clients are working with any recent Lustre release.