[LU-15828] o2iblnd: limit peer credits hiw at half the queue size

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Labels: o2iblnd
• Severity: 3
• Assignee: Serguei Smirnov
• Reporter: Serguei Smirnov

Description

O2iblnd "High watermark" parameter value is used to decide when it is time to return the accumulated credits to the peer:

 /* when eagerly to return credits */
 #define IBLND_CREDITS_HIGHWATER(t, conn) ((conn->ibc_version) == IBLND_MSG_VERSION_1 ? \
                                         IBLND_CREDIT_HIGHWATER_V1 : \
                         min(t->lnd_peercredits_hiw, (__u32)conn->ibc_queue_depth - 1))

If the values for the peer_credits and peercredits_hiw configured via module parameters are higher than on the other node, the code above may end up setting hiw too close to the negotiated queue depth. For example with peer_credits/peercredits_hiw at 32/16 vs 16/8 on the other end, the hiw will be set at 15 - with the queue depth of 16. 

This means that this node is holding on to credits for as long as possible and has the potential to cause the other node to run out of credits under load.

Issue Links

• is related to: LU-12569 IBLND_CREDITS_HIGHWATER does not check connection queue depth

Comments

[Peter Jones]

Landed for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49497/
Subject: LU-15828 o2iblnd: reset hiw proportionally
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e1944c29793d489429730a9445e243b448c3d751

[Gerrit Updater]

"Serguei Smirnov <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49497
Subject: LU-15828 o2iblnd: reset hiw proportionally
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 46cddb76a0fdc1532396ba83ed552ecc4fdc395f