[LU-15058] replace critical vmalloc allocations with genradix Created: 04/Oct/21  Updated: 14/Nov/22

Status: Open
Project: Lustre
Component/s: None
Affects Version/s: Lustre 2.15.0
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: James A Simmons Assignee: James A Simmons
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Any client or server running Lustre.

Issue Links:
Duplicate
is duplicated by LU-15606 sanity-quota test_8: osc_brw_prep_req... Resolved
Related
is related to LU-14469 lmv_rmfid() does 128K kmalloc() Open
is related to LU-13212 Lustre client hangs machine under mem... Resolved
Rank (Obsolete): 9223372036854775807

 Description   

When memory allocation grow top large the method of allocation moves from kmalloc to vmalloc. Using vmalloc() uses global locking which greatly degrades performance. For ORNL we ended up preventing users form creating files stripped greater than 512. This problem impact very large RPC and other components of Lustre. The solution is to use the generic_radix work present in newer kernel developed for this very issue.

 Comments   
Comment by Patrick Farrell [ 04/Oct/21 ]

Can you provide more references on how genradix trees are used to solve this memory allocation problem?

The description in the kernel docs doesn't suggest they're for this sort of problem:
"Generic radix trees/sparse arrays

Very simple and minimalistic, supporting arbitrary size entries up to PAGE_SIZE."[
https://www.kernel.org/doc/html/latest/core-api/generic-radix-tree.html]

Comment by James A Simmons [ 04/Oct/21 ]

See Linux commit ba20ba2e3743bac786dff777954c11930256075e. Reading the commit you will see generic radix trees is a direct replacement of flex_array. The link - https://www.kernel.org/doc/html/v4.14/core-api/flexible-arrays.html states the purpose of flexible arrays was to handle the described issue. Currently the only user is procfs. flexible arrays never had that many users.

Comment by Patrick Farrell [ 04/Oct/21 ]

Ah, OK - thanks.

Comment by Gerrit Updater [ 15/Oct/22 ]

"James Simmons <jsimmons@infradead.org>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/48883
Subject: LU-15058 osc: replace large RPC allocations with genradix
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: da4bf845b7e9a5de9ef44e874d434854994b41f7

Comment by Gerrit Updater [ 14/Nov/22 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/45890/
Subject: LU-15058 libcfs: introduce genradix support
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 841843f44cd16e2bfb9b31195f7398cff02f9088

Generated at Sat Feb 10 03:15:05 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.