Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20184

Extending Lustre NRS TBF Rate Limiting to Support Bandwidth (BPS) Mode

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      The Lustre NRS TBF (Token Bucket Filter) scheduler currently enforces per‑client rate limits in terms of IOPS (operations per second). 

      This model works well for controlling metadata or small, relatively uniform I/O. However, on OSS/OST servers with large and variable-sized I/O (e.g., parallel writes of multi‑MiB stripes), IOPS is often a poor proxy for “how much resource” a client is consuming:

      • A client issuing 100 x 4 KiB writes is much less demanding than one issuing 100 x 4 MiB writes.
      • Admins often want to bound bandwidth (bytes/s) per user/job/host, while still allowing bursts.

      To address this, we propose extending TBF to support bandwidth‑based throttling (BPS) in addition to the existing IOPS model.

      Add a bandwidth (BPS) rate mode to TBF:

      • Rules can specify rates in bytes per second (or equivalent units).
      • Enforcement is based on the actual I/O size of requests.

      The following command will start a rule in BPS performance mode:

       

      nrs_tbf_rule="start r1 projid={100} bps=5MB/s"
      nrs_tbf_rule="start r1 projid={100} bps=5MB"
      nrs_tbf_rule="start r1 projid={100} bps=5MiB/s"
      nrs_tbf_rule="start r1 projid={100} bps=5MiB"
      nrs_tbf_rule="start r1 projid={100} bps=5" # The default unit is MiB/s
      nrs_tbf_rule="start r1 projid={100} rate=5MiB/s"
      # explicit IOPS mode
      nrs_tbf_rule="start r1 projid={100} iops=100"
      

       

      ...

            qian_wc Qian Yingjin
            qian_wc Qian Yingjin
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: