[LU-4017] Add project quota support feature

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.10.0
• Labels: patch
• Assignee: Niu Yawei (Inactive)
• Reporter: Li Xi (Inactive)

Description

OST (or MDT) pool feature enables users to group OSTs together to make object placement more flexible which is a very useful mechanism for system management. However the pool support of quota is not completed now which limits the use of it. Luckily current quota framework is really powerful and flexible which makes it possible to add new extension.

We believe that pool support of quota will be helpful for a lot of use cases, so we are trying to complete it. With the help from the community, we've made some progress. The full patch will be a big one which involves quite a lot of components of Lustre. Any advice or feedback about the implementation will be really helpful.

Issue Links

• is cloned by: LU-11023 OST Pool Quotas
• is related to: LU-9554 upgrade to Lustre 2.10 breaks quota if project quota is not enabled
• is related to: LU-12056 tar doesn't support project id
• is related to: LU-12160 use-after-free in osd_object_delete()
• is related to: LUDOC-202 Lustre Manual Documentation for project quota feature
• is related to: LU-7991 Add project quota for ZFS
• is related to: LU-9555 "df /path/to/project" should return projid-specific values
• is related to: LU-9339 fix RHEL 7.2 project quota build error
• is related to: LU-9355 remove obsolete OBD_FL_LOCAL_MASK

Comments

[Wang Shilong (Inactive)]

Hi Nathan Rutman,

Your understanding is right, as-landed feature is project quota, not 'pool quota'.

[Nathan Rutman]

Description of this bug and early design docs talk about "pool quotas", as in the ability to administer quotas based on Lustre OST pools. The final landed feature seems to be about project quotas, not pool quotas - am I correct in stating that the as-landed feature cannot support pool quotas? I.e. I can't place a special quota on a set of SSD OSTs?

[Andreas Dilger]

One option would be to expose the projid value as a virtual xattr (e.g. "trusted.projid" or similar) from ext4/ldiskfs, so that it can be backed up and restored via getfattr/setfattr. This is also a problem for regular ext4 filesystems, so I would ask on the linux-ext4 mailing list to see what the agreement is there for implementing this.

[nasf (Inactive)]

I am thinking how to backup/restore the system among different backends, such as from ldiskfs to ZFS or the reverse case. I do not know whether there are other better solution for that.

[Li Xi (Inactive)]

Hi Fanyong,

I think backup/restore using tar will simply discards project ID. Project ID has an IOCTL with the name of EXT4_IOC_FSGETXATTR(FS_IOC_FSGETXATTR). And in order to support project ID backup, an ioctl of that type needs to be called. And as far as we've tested, that is not supported by tar. We need to push a patch to add that support.

 

BTW, I think there are better ways to backup MDT files than tar.

[nasf (Inactive)]

How to handle the project ID attribute via MDT file-level backup/restore? Originally, we can do that via tar/getfattr for backup, then untar/setfattr for restore, but it seems the project ID cannot be handled like that, right? Or I missed anything?