Details
-
Improvement
-
Resolution: Unresolved
-
Major
-
None
-
None
-
None
-
9223372036854775807
Description
Currently when we have a chokepoint of some sort (be it slow disk, a blocked ldlm resource and the like), relatively low number of threads we start by default gets consumed pretty quickly resulting in no requests being processed even those that would not block because they don't touch the contended resource.
While normally it's a good idea to maintain the thread pool small as it improves cpu cache effectiveness and also puts less of a load on a spinning disk setup (if present) when the entire thread pool is plugged for some time I think it is beneficial to actually spawn a lot more threads as a "one shot" only and we should not be any worse off wrt the end outcome. Memory consumption of server threads is relatively minor so even if you have a massive number of clients send you dozens of RPCs that results in tens of thousands of threads, modern systems should take it relatively easily.
Worst comes to worst all the new threads are also stuck, but some of them might actually progress which would be a net benefit.
Half measures are also possible to only spawn threads that would e.g. process "high priority" requests.
Attachments
Issue Links
- is duplicated by
-
LU-14556 Allow number of threads to grow when all existing threads are stuck
- Resolved
-
LU-14557 Allow number of threads to grow when all existing threads are stuck
- Resolved
-
LU-14558 Allow number of threads to grow when all existing threads are stuck
- Resolved
-
LU-14559 Allow number of threads to grow when all existing threads are stuck
- Resolved
-
LU-14560 Allow number of threads to grow when all existing threads are stuck
- Resolved
-
LU-14561 Allow number of threads to grow when all existing threads are stuck
- Resolved
-
LU-14562 Allow number of threads to grow when all existing threads are stuck
- Resolved
-
LU-14563 Allow number of threads to grow when all existing threads are stuck
- Resolved
- is related to
-
LU-17441 move rename RPC handling to MDS_IO_PORTAL
- Resolved