[LU-12815] Create multiple TCP sockets per SockLND

• Type: Technical task
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.15.0, Lustre 2.12.10
• Labels: performance
• Assignee: Amir Shehata (Inactive)
• Reporter: Andreas Dilger

Description

For high-bandwidth Ethernet interfaces (e.g. 100GigE), it would be useful to create multiple TCP connections per interface for bulk transfers in order to maximize performance (i.e. conns_per_peer=4 for socklnd in addition to o2iblnd). We already have three separate TCP connections per LND - read, write, and small message.

For large clusters this may be problematic because of the number of TCP connections to a server, but for smaller configurations this could be very useful.

Issue Links

• is duplicated by: LU-14293 Poor lnet/ksocklnd(?) performance on 2x100G bonded ethernet
• is related to: LU-14676 Better hash distribution to different CPTs when LNET router is exist

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/47252/
Subject: LU-12815 socklnd: add conns_per_peer parameter
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: be2c4bb928b5cf6b428d7974e8fd89ea177fa2df

[Gerrit Updater]

"Cyril Bordage <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/47252
Subject: LU-12815 socklnd: add conns_per_peer parameter
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: b370ffba7b778f9b5fee325a8c67228ca2454137

[Peter Jones]

Looks like everything has landed for 2.15

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/44417/
Subject: LU-12815 socklnd: set conns_per_peer based on link speed
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: c44afcfb72a1c2fd8392bfab3143c3835b146be6

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/41463/
Subject: LU-12815 socklnd: allow dynamic setting of conns_per_peer
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: a5cbe7883db6d77b82fbd83ad4c662499421d229

[Gerrit Updater]

Serguei Smirnov ([e-mail]) uploaded a new patch: https://review.whamcloud.com/44417
Subject: LU-12815 socklnd: set conns_per_peer based on link speed
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: ca2f4fed6d85d2e4506958fcc2e1c6c98eb2d020

[Andreas Dilger]

There may still be work needed to distribute RPCs from a single client to multiple CPTs on the server, in order to get the best performance for real IO workloads.

Otherwise, a client with a single interface (NID) will have all of its RPCs handled by cores in a single CPT, which is not quite the same as having multiple real interfaces on the client. Discussion is ongoing in LU-14676.

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/41056/
Subject: LU-12815 socklnd: add conns_per_peer parameter
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 71b2476e4ddb95aa42f4a0ea3f23b1826017bfa5