Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-17044

TBF: minimum guarantee RPC rate when the server is overloaded

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • None
    • 3
    • 9223372036854775807

    Description

      This seems to be something wanted from a long time: LU-8433
      The idea here is to be able to configure a rule like this:

      # lctl set_param ...=start fio jobid={fio.*} minrate=1000 rate=10000
# lctl get_param ...
regular requests:
CPT 0:
fio {fio.*} 1000-10000, ref 2
...

      The rate for each job matching "fio.*" should be between 1000-10000 RPC/s.
      So requests in queue belonging to a class bucket with rate below minimum acceptable rate should be scheduled first.

      Implementation:
      The main idea is to duplicate some part of TBF code to implement deadline and token for minimum rate.
      The compare function (tbf_cli_compare()) of the TBF binheap can be modify to firstly compare the deadlines of the minimum rate but only if the class has missed some token time slots before (current rate < minimum rate).
      That way, if a class is late (< minrate), it should be scheduled before the others class. If the class is not late anymore, it will be scheduled according its maximum rate.

      Realtime
      The realtime rule is used to prioritize the classes matching the rule when server loaded (if the sum of all the defined rate is above what the server can handle). The realtime classes will always try to match the specified rate by degrading the performances of the other classes.
      When a minimum rate is specified on a realtime rule, it makes sense to apply the "realtime" behavior on the lower rate limit of the classe.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. LU-17296 NRS TBF default rules

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-18183 shared rate limit for a TBF rule

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          is related to

          New Feature - A new feature of the product, which has yet to be developed. LU-8433 Maximizing Bandwidth utilization by TBF Rule with Dependency

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            [LU-17044] TBF: minimum guarantee RPC rate when the server is overloaded
            npalsetia Nushafreen Palsetia made changes -
            Link New: This issue is related to DDN-6060 [ DDN-6060 ]
            adilger Andreas Dilger made changes -
            Summary Original: TBF: minimum guarantee RPC rate when the serveur is overloaded New: TBF: minimum guarantee RPC rate when the server is overloaded
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-18183 [ LU-18183 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-8433 [ LU-8433 ]
            mrasobarnett Matt Rásó-Barnett made changes -
            Link New: This issue is related to EXR-428 [ EXR-428 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-17296 [ LU-17296 ]
            gerrit Gerrit Updater added a comment -

            "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52038
            Subject: LU-17044 tbf: implement minrate for tbf rules
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 07bdc9c2cdad6937cfc9839926c49d0186901a52

            gerrit Gerrit Updater added a comment - "Etienne AUJAMES <eaujames@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/52038 Subject: LU-17044 tbf: implement minrate for tbf rules Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 07bdc9c2cdad6937cfc9839926c49d0186901a52
            pjones Peter Jones made changes -
            Assignee Original: WC Triage [ wc-triage ] New: Etienne Aujames [ eaujames ]
            eaujames Etienne Aujames made changes -
            Description Original: This seems to be something wanted from a long time: LU-8433
            The idea here is to be able to configure a rule like this:
            {noformat}
            # lctl set_param ...=start fio jobid={fio.*} minrate=1000 rate=10000
            # lctl get_param ...
            regular requests:
            CPT 0:
            fio {fio.*} 1000-10000, ref 2
            ...
            {noformat}
            The rate for each job matching "fio.*" should be between 1000-10000 RPC/s.
            So requests in queue belonging to a class bucket with rate below minimum acceptable rate should be scheduled first.

            *Implementation:*
            The main idea is to duplicate some part of TBF code to implement deadline and token for minimum rate.
            The compare function (tbf_cli_compare()) of the TBF binheap can be modify to firstly compare the deadlines of the minimum rate but only if the class has missed some token time slots before (current rate < minimum rate).
            That way, if a class is late (< minrate), it should be scheduled before the others class. If the class is not late anymore, it will be scheduled according its maximum rate.

            *Realtime*
            The realtime rule is used to prioritize the classes matching the rule when server loaded (if the sum of all the defined rate is above what the server can handle). The realtime classes will always try to match the specified rate by degrading the performances of the other classes.
            When a minimum rate is specified on a realtime rule, it makes sense to apply the "realtime" behavior on the lower rate limit of the classes.             		New: This seems to be something wanted from a long time: LU-8433
            The idea here is to be able to configure a rule like this:
            {noformat}
            # lctl set_param ...=start fio jobid={fio.*} minrate=1000 rate=10000
            # lctl get_param ...
            regular requests:
            CPT 0:
            fio {fio.*} 1000-10000, ref 2
            ...
            {noformat}
            The rate for each job matching "fio.*" should be between 1000-10000 RPC/s.
            So requests in queue belonging to a class bucket with rate below minimum acceptable rate should be scheduled first.

            *Implementation:*
            The main idea is to duplicate some part of TBF code to implement deadline and token for minimum rate.
            The compare function (tbf_cli_compare()) of the TBF binheap can be modify to firstly compare the deadlines of the minimum rate but only if the class has missed some token time slots before (current rate < minimum rate).
            That way, if a class is late (< minrate), it should be scheduled before the others class. If the class is not late anymore, it will be scheduled according its maximum rate.

            *Realtime*
            The realtime rule is used to prioritize the classes matching the rule when server loaded (if the sum of all the defined rate is above what the server can handle). The realtime classes will always try to match the specified rate by degrading the performances of the other classes.
            When a minimum rate is specified on a realtime rule, it makes sense to apply the "realtime" behavior on the lower rate limit of the classe.
            eaujames Etienne Aujames made changes -
            Description Original: This seems to be something wanted from a long time: LU-8433
            The idea here is to be able to configure a rule like this:
            {noformat}
            # lctl set_param ...=start fio jobid={fio.*} minrate=1000 rate=10000
            # lctl get_param ...
            regular requests:
            CPT 0:
            fio {fio.*} 1000-10000, ref 2
            ...
            {noformat}
            The rate for each job matching {fio.*} should be between 1000-10000 RPC/s.
            So requests in queue belonging to a class bucket with rate below minimum acceptable rate should be scheduled first.

            *Implementation:*
            The main idea is to duplicate some part of TBF code to implement deadline and token for minimum rate.
            The compare function (tbf_cli_compare()) of the TBF binheap can be modify to firstly compare the deadlines of the minimum rate but only if the class has missed some token time slots before (current rate < minimum rate).
            That way, if a class is late (< minrate), it should be scheduled before the others class. If the class is not late anymore, it will be scheduled according its maximum rate.

            *Realtime*
            The realtime rule is used to prioritize the classes matching the rule when server loaded (if the sum of all the defined rate is above what the server can handle). The realtime classes will always try to match the specified rate by degrading the performances of the other classes.
            When a minimum rate is specified on a realtime rule, it makes sense to apply the "realtime" behavior on the lower rate limit of the classes.             		New: This seems to be something wanted from a long time: LU-8433
            The idea here is to be able to configure a rule like this:
            {noformat}
            # lctl set_param ...=start fio jobid={fio.*} minrate=1000 rate=10000
            # lctl get_param ...
            regular requests:
            CPT 0:
            fio {fio.*} 1000-10000, ref 2
            ...
            {noformat}
            The rate for each job matching "fio.*" should be between 1000-10000 RPC/s.
            So requests in queue belonging to a class bucket with rate below minimum acceptable rate should be scheduled first.

            *Implementation:*
            The main idea is to duplicate some part of TBF code to implement deadline and token for minimum rate.
            The compare function (tbf_cli_compare()) of the TBF binheap can be modify to firstly compare the deadlines of the minimum rate but only if the class has missed some token time slots before (current rate < minimum rate).
            That way, if a class is late (< minrate), it should be scheduled before the others class. If the class is not late anymore, it will be scheduled according its maximum rate.

            *Realtime*
            The realtime rule is used to prioritize the classes matching the rule when server loaded (if the sum of all the defined rate is above what the server can handle). The realtime classes will always try to match the specified rate by degrading the performances of the other classes.
            When a minimum rate is specified on a realtime rule, it makes sense to apply the "realtime" behavior on the lower rate limit of the classes.

            People

              eaujames Etienne Aujames
              eaujames Etienne Aujames
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: