Details
-
Improvement
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
-
3
-
9223372036854775807
Description
We already have batched RPC support for statahead since 2.16, and timday raises a very valid quiestion why don't we use it for more than statahead.
Now normally there's a latency hit if we wait up for other contenders and that's kind of hard to know what makes sense there for the right balance.
But there's one clear case that we can take advantage of it: when we have so many single RPCs queued that all RPC slots for a target are exhausted, we really should start batching all the waiting ones up.
Something along the lines of a "waiting bus/train RPC". It sits there at the stop waiting for an RPC slot to free up and until that time all new outgoing nonmodifying RPCs would just go onto this "bus". The moment the slot opens up, "the doors are closing" and the "Bus" is launching onto the network, as another now empty "bus" arrives at the station.
This should be relatively easy to implement I suspect. And then after that we can try to reduce max RPCs in flight down vs the usual "up" direction and see how well that fares.
Now we do have similar waiting mechanism for modifyign RPCs and the number of those is typically much smaller, but I suspect having multiple modifyign RPCs is going to be much trickier so we are not covering this part here.
