Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-14564

Allow number of threads to grow when all existing threads are stuck

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • None
    • None
    • None
    • 9223372036854775807

    Description

      Currently when we have a chokepoint of some sort (be it slow disk, a blocked ldlm resource and the like), relatively low number of threads we start by default gets consumed pretty quickly resulting in no requests being processed even those that would not block because they don't touch the contended resource.

      While normally it's a good idea to maintain the thread pool small as it improves cpu cache effectiveness and also puts less of a load on a spinning disk setup (if present) when the entire thread pool is plugged for some time I think it is beneficial to actually spawn a lot more threads as a "one shot" only and we should not be any worse off wrt the end outcome. Memory consumption of server threads is relatively minor so even if you have a massive number of clients send you dozens of RPCs that results in tens of thousands of threads, modern systems should take it relatively easily.

      Worst comes to worst all the new threads are also stuck, but some of them might actually progress which would be a net benefit.

      Half measures are also possible to only spawn threads that would e.g. process "high priority" requests.

      Attachments

        Issue Links

          Activity

            People

              wc-triage WC Triage
              green Oleg Drokin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: