[LU-4017] Add project quota support feature

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.10.0
• Labels: patch
• Assignee: Niu Yawei (Inactive)
• Reporter: Li Xi (Inactive)

Description

OST (or MDT) pool feature enables users to group OSTs together to make object placement more flexible which is a very useful mechanism for system management. However the pool support of quota is not completed now which limits the use of it. Luckily current quota framework is really powerful and flexible which makes it possible to add new extension.

We believe that pool support of quota will be helpful for a lot of use cases, so we are trying to complete it. With the help from the community, we've made some progress. The full patch will be a big one which involves quite a lot of components of Lustre. Any advice or feedback about the implementation will be really helpful.

Issue Links

• is cloned by: LU-11023 OST Pool Quotas
• is related to: LU-9554 upgrade to Lustre 2.10 breaks quota if project quota is not enabled
• is related to: LU-12056 tar doesn't support project id
• is related to: LU-12160 use-after-free in osd_object_delete()
• is related to: LUDOC-202 Lustre Manual Documentation for project quota feature
• is related to: LU-7991 Add project quota for ZFS
• is related to: LU-9555 "df /path/to/project" should return projid-specific values
• is related to: LU-9339 fix RHEL 7.2 project quota build error
• is related to: LU-9355 remove obsolete OBD_FL_LOCAL_MASK

Comments

[Li Xi (Inactive)]

Hi Fanyong,

I think backup/restore using tar will simply discards project ID. Project ID has an IOCTL with the name of EXT4_IOC_FSGETXATTR(FS_IOC_FSGETXATTR). And in order to support project ID backup, an ioctl of that type needs to be called. And as far as we've tested, that is not supported by tar. We need to push a patch to add that support.

 

BTW, I think there are better ways to backup MDT files than tar.

[nasf (Inactive)]

How to handle the project ID attribute via MDT file-level backup/restore? Originally, we can do that via tar/getfattr for backup, then untar/setfattr for restore, but it seems the project ID cannot be handled like that, right? Or I missed anything?

[Peter Jones]

Landed for 2.10

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26577/
Subject: LU-4017 quota: cleanup to improve quota codes
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4ce3219eb8e6a07c5c37e4b425b29195488005c3

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26411/
Subject: LU-4017 quota: extend to test project quota
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 0eff45318562be759517bc7702365dfd450e518d

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26464/
Subject: LU-4017 quota: add project id support to lfs find
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 3dad616e09fc2a89174900a4d4dbb60308650418

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26463/
Subject: LU-4017 quota: add project inherit attributes
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 2e92c57d71384d50f06baf5b6591d2809c8288b2

[Gerrit Updater]

Andreas Dilger ([e-mail]) merged in patch https://review.whamcloud.com/26581/
Subject: LU-4017 tune2fs: fix BUGs of tuning project quota
Project: tools/e2fsprogs
Branch: master-lustre
Current Patch Set:
Commit: a21925594cd18569f6618c02ea7b28c4afab477b

[Gerrit Updater]

Wang Shilong ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26621
Subject: LU-4017 tune2fs: fix BUGs of tuning project quota
Project: tools/e2fsprogs
Branch: master-lustre
Current Patch Set: 1
Commit: e51dd021fa64a916e03df8dadf7def598e1f2529

[James A Simmons]

Since RHEL tends to use 5 year old stacks we are looking at 2021 when we don't need patched kernels

[Shuichi Ihara (Inactive)]

Yes. That is true at this point. We are scoping out what is required to extend this work further.

Exactly. SInce all ldiskfs (ext4) and kernel patches were merged in upstream linux kernel (linux-4.5), once lustre server supprots 4.5 kernel or above, we don't need patched server for project quota anymore. Meantime, ldisfks patches are still backportable for other kernel e.g. SLES12SP2 in the future work.
Also, once zfs supports project id, project quota will be extanable to supoprt osd-zfs.