[LU-1144] implement a NUMA aware ptlrpcd binding policy

• Type: Improvement
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.3.0
• Environment: lustre 2.1 with patch ORNL-22, machine with several NUMA nodes
• Assignee: Andreas Dilger
• Reporter: Gregoire Pichon

Description

The binding policy implemented by the ORNL-22 patch does not take care of NUMA aspects. It means ptlrpcd thread partners can be in different NUMA nodes. When the io requests will be processed by ptlrpcd threads in their affinity domain it will be more performant to have ptlrpcd partners in the same NUMA domain.

Comments

[Jay Lan (Inactive)]

Not sure if this "NUMA aware" patch helps Lustre BZ-23047 "zone_reclaim_mode = 1 tanks lustre performance on clients", so please advise. Appreciated!

My understanding of the BZ-23047 problem was that zone_reclaim tried to free cache memory aggressively but failed since Lustre held a lock of those clean cache. Without freeing those clean cache, memory allocation may have to grab from nodes of some distance.

[Peter Jones]

Landed for 2.3

[Build Master (Inactive)]

Integrated in lustre-dev » x86_64,client,el6,inkernel #340
LU-1144 ptlrpc: implement a NUMA aware ptlrpcd binding policy (Revision db8a6829cb1c9869d998920c095a14de2d5300bf)

Result = SUCCESS
Oleg Drokin : db8a6829cb1c9869d998920c095a14de2d5300bf
Files :

• lustre/include/lustre_net.h
• lustre/ptlrpc/ptlrpcd.c

[Build Master (Inactive)]

Integrated in lustre-dev » x86_64,server,el5,inkernel #340
LU-1144 ptlrpc: implement a NUMA aware ptlrpcd binding policy (Revision db8a6829cb1c9869d998920c095a14de2d5300bf)

Result = SUCCESS
Oleg Drokin : db8a6829cb1c9869d998920c095a14de2d5300bf
Files :

• lustre/include/lustre_net.h
• lustre/ptlrpc/ptlrpcd.c

[Build Master (Inactive)]

Integrated in lustre-dev » i686,client,el5,inkernel #340
LU-1144 ptlrpc: implement a NUMA aware ptlrpcd binding policy (Revision db8a6829cb1c9869d998920c095a14de2d5300bf)

Result = SUCCESS
Oleg Drokin : db8a6829cb1c9869d998920c095a14de2d5300bf
Files :

• lustre/ptlrpc/ptlrpcd.c
• lustre/include/lustre_net.h

[Build Master (Inactive)]

Integrated in lustre-dev » x86_64,server,el6,inkernel #340
LU-1144 ptlrpc: implement a NUMA aware ptlrpcd binding policy (Revision db8a6829cb1c9869d998920c095a14de2d5300bf)

Result = SUCCESS
Oleg Drokin : db8a6829cb1c9869d998920c095a14de2d5300bf
Files :

• lustre/include/lustre_net.h
• lustre/ptlrpc/ptlrpcd.c

[Build Master (Inactive)]

Integrated in lustre-dev » i686,server,el5,inkernel #340
LU-1144 ptlrpc: implement a NUMA aware ptlrpcd binding policy (Revision db8a6829cb1c9869d998920c095a14de2d5300bf)

Result = SUCCESS
Oleg Drokin : db8a6829cb1c9869d998920c095a14de2d5300bf
Files :

• lustre/ptlrpc/ptlrpcd.c
• lustre/include/lustre_net.h

[Build Master (Inactive)]

Integrated in lustre-dev » i686,client,el6,inkernel #340
LU-1144 ptlrpc: implement a NUMA aware ptlrpcd binding policy (Revision db8a6829cb1c9869d998920c095a14de2d5300bf)

Result = SUCCESS
Oleg Drokin : db8a6829cb1c9869d998920c095a14de2d5300bf
Files :

• lustre/include/lustre_net.h
• lustre/ptlrpc/ptlrpcd.c

[Build Master (Inactive)]

Integrated in lustre-dev » x86_64,client,el5,inkernel #340
LU-1144 ptlrpc: implement a NUMA aware ptlrpcd binding policy (Revision db8a6829cb1c9869d998920c095a14de2d5300bf)

Result = SUCCESS
Oleg Drokin : db8a6829cb1c9869d998920c095a14de2d5300bf
Files :

• lustre/ptlrpc/ptlrpcd.c
• lustre/include/lustre_net.h

[Build Master (Inactive)]

Integrated in lustre-master » i686,client,el6,inkernel #483
LU-1144 ptlrpc: implement a NUMA aware ptlrpcd binding policy (Revision db8a6829cb1c9869d998920c095a14de2d5300bf)

Result = SUCCESS
Oleg Drokin : db8a6829cb1c9869d998920c095a14de2d5300bf
Files :

• lustre/include/lustre_net.h
• lustre/ptlrpc/ptlrpcd.c