[LUDOC-189] Document LNET Message Credits

• Status: Open
• Project: Lustre Documentation
• Type: New Feature
• Priority: Minor
• Reporter: Linda Bebernes (Inactive)
• Assignee: Amir Shehata (Inactive)
• Resolution: Unresolved

Description

-->> This ticket (Bugzilla 24404 https://bugzilla.lustre.org/show_bug.cgi?id=24404) also requests that any “special considerations in routed configurations for
other parameters like LND credits, peer_credits, ntx, etc.,” be documented.

[AMIR] - Here is the sum of information I have gathered so far about credits.

credits: Each Network Interface maintains the maximum number of credits. This can be configured by the user. It indicates the maximum number of messages that can be transmitted out of that NI simultaneously.
e.g, if NI (ib0) credits is 128, then you can send up to 128 messages to ib0, if you send 8 messages to each peer, then total 16 peers.
Peer_credits: is how many concurrent messages a peer can send
Router buffer credits: is how many buffers can be taken as router buffer for a peer.
Peer_Min_credits: is a historical data that shows the lowest number of credits ever.
each peer will record: a) max credits it can have and it's const, b) current tx credits, c) lowest credits eve
(max_peer_credits - max(0, tx_credits)) is message in-flight, if tx_credits is negative, then the negative value is number of queued messages.
—

Hi Linda,

These sections talk about the credit parameters which can be configured. I was thinking that there is more of a full description of the entire credit system that LNET uses, which is not currently documented. This will require more work, because I’ll need to go through the code and understand it a bit more, do a write up on it pass it by Doug and Liang. This process I think will take a while. That’s why I agree with Richard’s recommendation to track it separately.

I don’t feel that adding this information in 25.3.4 is sufficient to give a clear picture of the credit system.

Thanks
amir

Comments

[Kit Westneat (Inactive)]

In the past, there have been some options that have to be enabled on all nodes, as opposed to those that can be enabled on just a subset. It would be good to know which can be enabled on a subset of the nodes as opposed to all of them.