[LU-17880] cpu_npartition is only honored in a single NUMA system

• Type: Bug
• Resolution: Unresolved
• Priority: Minor
• Severity: 3
• Assignee: Di Wang
• Reporter: Di Wang

Description

cpu_npartition module parameter is only honored in single NUMA system, so in a multiple numa system, it will always use default cpu partition for ptlrpc services, which has too much cores in each partition, and it may impact the performance.

Here is a perf trace result I got from an environment with 45 cores in one CPT, where more than 40% percent of CPU cycles being spent in spin lock.

"
+ 93.19% 0.00% [k] ret_from_fork - - ◆
+ 93.19% 0.00% [k] kthread - - ▒
+ 73.54% 0.48% [k] ptlrpc_main - - ▒
+ 41.70% 41.40% [k] native_queued_spin_lock_slowpath - - ▒
+ 37.44% 0.14% [k] ptlrpc_server_handle_request - - ▒
+ 33.91% 0.15% [k] tgt_request_handle - - ▒
"
Anyway I will post a patch to set the number of partition in a multiple NUMA environment, and also discuss how many cores in each partition by default.

Comments

[Gerrit Updater]

"wangdi <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/55209
Subject: LU-17880 libcfs: set cpu_npartition in a multiple NUMA system.
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 5300bd24729cc7f6f6b7ecb0e835aa40b661cad5