[LU-15828] o2iblnd: limit peer credits hiw at half the queue size

Related — Sat, 21 Jan 2023 01:28:03 +0000

O2iblnd "High watermark" parameter value is used to decide when it is time to return the accumulated credits to the peer:

 /* when eagerly to return credits */
 #define IBLND_CREDITS_HIGHWATER(t, conn) ((conn->ibc_version) == IBLND_MSG_VERSION_1 ? \
                                         IBLND_CREDIT_HIGHWATER_V1 : \
                         min(t->lnd_peercredits_hiw, (__u32)conn->ibc_queue_depth - 1))

If the values for the peer_credits and peercredits_hiw configured via module parameters are higher than on the other node, the code above may end up setting hiw too close to the negotiated queue depth. For example with peer_credits/peercredits_hiw at 32/16 vs 16/8 on the other end, the hiw will be set at 15 - with the queue depth of 16.

This means that this node is holding on to credits for as long as possible and has the potential to cause the other node to run out of credits under load.

Whamcloud Community JIRA

[LU-15828] o2iblnd: limit peer credits hiw at half the queue size