Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-18222

Implement quota aggregation limits by ID range

Details

    • Bug
    • Resolution: Unresolved
    • Minor
    • None
    • None

    Description

      A number of sites have requested the ability to have hierarchical/nested project quotas in order to aggregate multiple project IDs for a tenant into a single aggregate limit, in order to restrict the total usage for multiple projects.

      Rather than implement this at the level of the backing filesystem (which doesn't currently have space in the inode to store more IDs), one option would be for the MDS to add the quota usage from multiple project IDs into a single total, in a manner similar to OST pool quotas, and in general how the MDS is managing quota limits rather than quota accounting.

      The MDS would sum the quota usage across all IDs in the aggregate, and then decide whether IDs would be granted new qunits when the OSTs have consumed their total space.

      Some open questions include:

      • what is a good name for this feature? "aggregate quota", "parent quota", "collective quota", other? "group quota" is already used.
      • should this allow aggregating quotas from multiple/random disjoint project IDs? would it be enough to allow a range of IDs (e.g. 1000-1999 or 100000-199999) to be aggregated into single parent quota limit
      • should an actual project ID (e.g. 1000 or 100000) be allocated as the placeholder for the "aggregate" quota accounting/limits? or should this grouping be "virtual" without an actual ID (which might cause confusion between the actual "p=1000" usage and the aggregate usage)
      • should it be possible to aggregate a hierarchy of projects (e.g. 1000-1999 in one parent A, 3000-3999 into a second parent B, then add "A" and "B" into a third aggregate)? it may be that we can add this afterward on an as-needed basis, if it isn't easily done in the initial implementation

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. LU-16857 OST object allocation should not select OSTs/pools where quota is exceeded

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. LU-17166 add NRS TBF rule for projid

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open
          is related to

          Bug - A problem which impairs or prevents the functions of the product. LU-16771 add client statfs() cache for project quotas

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Reopened

          New Feature - A new feature of the product, which has yet to be developed. LU-18109 Nodemap UID/GID/PROJID idmap offsetting

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LU-11571 MDT pool

          • Major - Major loss of function.
          • Open
          mentioned in

          Page Loading...

          Activity

            [LU-18222] Implement quota aggregation limits by ID range
            chunter-wc Chris Hunter made changes -
            Link Original: This issue is related to EXR-530 [ EXR-530 ]
            chunter-wc Chris Hunter made changes -
            Link New: This issue is related to EXR-656 [ EXR-656 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-16857 [ LU-16857 ]
            sebastien Sebastien Buisson made changes -
            Labels Original: quota New: hard quota
            sebastien Sebastien Buisson made changes -
            Link Original: This issue is related to EX-11327 [ EX-11327 ]
            sebastien Sebastien Buisson made changes -
            Epic Link New: EX-11327 [ 86351 ]
            sebastien Sebastien Buisson made changes -
            Assignee Original: WC Triage [ wc-triage ] New: Sergey Cheremencev [ scherementsev ]
            mrasobarnett Matt Rásó-Barnett made changes -
            Remote Link New: This issue links to "Page (Whamcloud Community Wiki)" [ 40842 ]
            sebastien Sebastien Buisson made changes -
            Link New: This issue is related to EX-11327 [ EX-11327 ]
            gerrit Gerrit Updater added a comment -

            "Sergey Cheremencev <scherementsev@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57269
            Subject: LU-18222 quota: project Pool Quotas POC
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: ad39a65d13a625d68f1a3c9e2c57a3f351d9f361

            gerrit Gerrit Updater added a comment - "Sergey Cheremencev <scherementsev@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57269 Subject: LU-18222 quota: project Pool Quotas POC Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: ad39a65d13a625d68f1a3c9e2c57a3f351d9f361

            People

              scherementsev Sergey Cheremencev
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated: