[LU-10776] Large grant requests still don't work, resulting in small write RPCs

• Status: Resolved
• Project: Lustre
• Fix Version/s: Lustre 2.12.0, Lustre 2.10.7
• Type: Bug
• Priority: Minor
• Reporter: Patrick Farrell (Inactive)
• Assignee: Patrick Farrell (Inactive)
• Resolution: Fixed
• Labels: patch
• Severity: 3

Description

Previously, LU-2049 introduced a bug with older clients and larger grant requests stemming from 4 MiB RPCs. This was resolved by a simple tweak to interop behavior, but unfortunately, there are still (at least two...) bugs in the grant handling code with 16 MiB RPCs.

Specifically, the server automatically refuses grant requests > 2 GiB in size, which is a problem because client side grant requested is calculated like this:
bytes_per_page*pages_per_rpc*max_rpcs_in_flight + various padding (extent tax, etc)
(see osc_announce_cached)

Consider 16 MiB RPCs with 128 RPCs in flight:
4096*4096*128 = 2 GiB, and that + tax is > 2 GiB. This gets those requests refused by the server, which results in terrible performance.

There is a different but also "fun" bug when we go higher:
16 MiB RPCs with 256 RPCs in flight:
4096*4096*256 = 4 GiB + tax is > 4 GiB.... Which overflows the 32 bit unsigned o_undirty which is used to communicate grant request to the server. So we actually request, effectively, only the extent tax and other padding. This results in a maximum grant request of ~ 19 MiB, which works quite badly.

The result of all this is badly degraded performance at very high RPC sizes and max_rpcs_in_flight.

Suggested patch will limit grant request to < 2 GiB.

Comments

[Gerrit Updater]

Patrick Farrell ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31533
Subject: LU-10776 osc: Do not request more than 2GiB grant
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 329aa190e3790c1485c0622fc5d0a110918f5d56

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/31533/
Subject: LU-10776 osc: Do not request more than 2GiB grant
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: c0246d8878097a618efa9296d90896ac2cc2e9e4

[Peter Jones]

Landed for 2.12

[Gerrit Updater]

Li Xi ([e-mail]) uploaded a new patch: https://review.whamcloud.com/34051
Subject: LU-10776 osc: Do not request more than 2GiB grant
Project: fs/lustre-release
Branch: b2_10
Current Patch Set: 1
Commit: 3accbc1867e03375e82718ab37b31be967e4a757

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/34051/
Subject: LU-10776 osc: Do not request more than 2GiB grant
Project: fs/lustre-release
Branch: b2_10
Current Patch Set:
Commit: 5ad464969157aee2d868f09a4dc302e01d0221c6