[LU-18635] add non-page tunable max_mb_per_rpc parameter

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Labels: easy
• Severity: 3
• Assignee: Rahul Bansal
• Reporter: Andreas Dilger

Description

It would be useful to add a new "{osc,mdc}.*.max_mb_per_rpc" tunable parameter to dissociate the RPC size from the client page size as implied by "{osc,mdc}.*.max_pages_per_rpc". While it is already to specify "max_pages_per_rpc" with a unit like "4M" or "16M" to avoid issues with page sizes, this is still printed as a number of pages, and values without units are assumed to be pages. It would be better to have a dedicated parameter for specifying the RPC size, and (very) slowly deprecate max_pages_per_rpc in the future.

The max_mb_per_rpc parameter should internally convert the specified parameter to pages and still modify the same cl_max_pages_per_rpc variable, in addition to the existing max_pages_per_rpc parameter, but will provide a more consistent userspace interface for doing so.

The parameter should accept units and fractional parameters (e.g. 0.5M and 64k), which is already handled by sysfs_memparse(), so that it is possible to specify RPC sizes smaller than 1MiB in the rare case this is needed.

Issue Links

• is duplicated by: LU-18634 add non-page tunable max_mb_per_rpc parameter
• is related to: LU-18739 max_pages_per_rpc gives "out of range" error for values less than 1M
• is related to: LU-10463 Poor write performance periodically on repeated test runs

Comments

[Andreas Dilger]

It looks like the default obdfilter.*.brw_size=1 for ZFS since patch https://review.whamcloud.com/30757 ("LU-10463 osd-zfs: use 1MB RPC size by default"), so this is "working as expected".

It is probably sufficient to test this only with ldiskfs (which runs for every patch), and just skip the part that is setting max_mb_per_rpc > 20 if $(import_param $tgt max_brw_size) < 4194304.

[Rahul Bansal]

I am facing a problem in setting "max_pages_per_rpc" parameter in ZFS.
I found this document (https://wiki.lustre.org/ZFS_recordsize_Property) mentioning that RPC size for ZFS ranges between 512B to 1M, theoretical.
On my local setup, value for "max_pages_per_rpc" is stuck at 256 (1M). I was not able to change it to any other value using "lctl".
Because of this "49b" testcase that I added in the PR (https://review.whamcloud.com/c/fs/lustre-release/+/58095) is failing.
Failure logs: https://testing.whamcloud.com/gerrit-janitor/49375/results.html

[Gerrit Updater]

"Rahul Bansal <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58095
Subject: LU-18635 sysfs: add non-page tunable max_mb_per_rpc parameter
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 20efb8b1024c9330c53b9e1f0d073997fe482cd2

[James A Simmons]

The kernel does not support float. The closest to support showing "float" to the user is to use the kernels string_get_size() in the kernel.

[Rahul Bansal]

While trying to show the value of "max_mb_per_rpc" in float, I am getting "error: SSE register return with SSE disabled" error.
I found two solutions to this problem, either we can use fixed-point format or we can provide "max_bytes_per_rpc" parameter instead of "max_mb_per_rpc".
Can you please verify what approach should I take? Or is there some other standard approach?