Shilong
Could you please advise?
Thanks
Peter
This could be a little tricky, currently Lustre did what ext4/xfs does:
int ll_ioctl_check_project(struct inode *inode, struct fsxattr *fa)
{
/*
* Project Quota ID state is only allowed to change from within the init
* namespace. Enforce that restriction only if we are trying to change
* the quota ID state. Everything else is allowed in user namespaces.
*/
if (current_user_ns() == &init_user_ns)
return 0;
Since you are switching to sthiell from root, it is in still in same namespace, i guess you could
try ssh sthiell@fir-io7-s2 and then do same operations which it will return no permissions errors.
Hi Shilong,
Thanks for your reply! This sounded a good point at first, so I tried without su, with ssh sthiell@fir-io7-s2 but same thing: I was able to modify and even clear the project ID as a user. You said it will return "no permissions errors", actually we want that it returns an error in that case (something like permission denied). Did we miss anything else?
Yup, you are right.
The point is it looks current check is broken somehow.
Following check always return true in Centos7 client:
current_user_ns() == &init_user_ns
Will check more with this.
This is a bit interesting, currently XFS/ext4 did same thing for this, that owner could change project ID of files, following is testing results of XFS:
[wangsl@server wangsl]$ chattr -p 1 foo
[wangsl@server wangsl]$ lsattr -p foo
1 ------------------- foo
[wangsl@server wangsl]$ chattr -p 2 foo
[wangsl@server wangsl]$ lsattr -p foo
2 ------------------- foo
[wangsl@server wangsl]$ df . -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda4 xfs 48G 82M 48G 1% /mnt
[wangsl@server wangsl]$ uname -r
5.1.17-300.fc30.x86_64
This is a bit different behavior than originally we think for lustre project quota use case, we could
add ROOT CAP check for lustre for surely.
Hi Shilong,
Thanks for checking! It's a bit of a disappointment,, as projects quotas don't seem to be usable as-is in a shared environment context. If users can easily circumvent project quotas by simply changing the project id on their files, there's not much point in enforcing project quotas in the first place.
Now, for accounting purposes, there probably are valid use cases to allow users to set project ids on specific directories, and avoid having to rely on `du` to get the total volume/inode usage of a given file hierarchy for instance, project quotas would give that information right away.
So maybe there's a need for a more flexible tunable? To make changing project ids on files and directory either user-accessible, or a root-only action, depending on the use case?
To be honest, I'm a bit stumped by all of this: how are people using project quotas in practice (not only with Lustre, but with XFS or ext4 as well) if regular users can switch project ids as they wish?
In any case, thanks a lot for looking into it!
Cheers,
--
Kilian
It probably makes sense to raise this as a question on the linux-fsdevel and/or linux-ext4 mailing lists. It is probably best to not mention Lustre and instead focus on the behavior of ext4 and/or xfs on the lists.
Yup, let me send an email to upstream to raise this problem.
I just raised this problem in the upstream linux mail list:
After understanding the reasons of the project ID behavior in the mail list, I still think the current behavior (not allowing the file owner to change the project ID of its file) is still useful in a lot of use cases. It would be a lose to change the current Lustre behavior to the same with Ext4/XFS, especially considering that most likely Lustre are used in the circumstances with more users than local file systems. Another factor is Lustre usually have much larger size than local file systems. That means if any user walks around the project quota limitations, it could troubles to the administrators. Thus, I don't think it is proper to change the current bahvior of Lustre project quota .
Nevertheless, it is still valuable to enable some use cases of project quota similar to Ext4/XFS, especailly when containers are being used. Thus, I think it might be a good idea to add a new mount option to Lustre client like "prjid_owner_changable". If that option is enabled whem mounting, then it will have the same behavior like Ext4/XFS.
In this way, I think we are enabling all kinds of use cases depending on the requirement. It is a question whether "prjid_owner_changable" should be enabled by default or not. I feel the answer is "no" at least for now.
Let me push a patch to limit root for project change in default, and with prjid_owner_changeable option, users could change their project ID and inherit flags.
I think rather than a mount option, it would be better to have a tunable on the MDS to control this, maybe along with a nodemap option as Sébastien has done in patch https://review.whamcloud.com/36433 so that it can be controlled by the server instead of the client, which may be a VM under the control of an unfriendly user.
PS: I agree that regular users should not be able to change the projid by default.
and I agree with Andreas 
> rather than a mount option, it would be better to have a tunable on the MDS to control this
Agreed on that.
Hi Li Xi,
Just to clarify, when you say:
> I still think the current behavior (not allowing the file owner to change the project ID of its file) is still useful in a lot of use cases.
It is actually the opposite: currently in Lustre the file owner can change the project ID of its file. That means on filesystems where project quotas are enforced, users can likely bypass their project quota if they use lfs project or chattr -p (if available on the cluster) on their own files. We need a fix to only allow admins to set/change/remove project IDs and this should be the default, as in my understanding, the most common use case with Lustre is to use project quotas to enforce some kind of directory quotas (with inheritance), managed by system administrators, not users.
But a tunable on the server should solve that.
I think it makes sense to add a tunable on the MDS similar to "mdt.*.enable_remote_dir_gid" that allows users in specific groups to also change the projid. By default, I think it makes sense to default to only allow root (CAP_SYS_RESOURCE) to change the projid directly, and set "mdt.*.change_projid_gid=0". Any users in that group (e.g. wheel or admin) can also change projid directly, if it is "-1" then any user can change it.
Wang Shilong (wshilong@ddn.com) uploaded a new patch: https://review.whamcloud.com/36544
Subject: LU-12826 mdt: limit root to change project state
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 3d38465be14aa28b89f75384980c4e96e15a1f04
It would be nice to have a backport of this important patch to b2_12 when it has landed (I tried to apply it but it doesn't work as is). Thanks much!
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36544/
Subject: LU-12826 mdt: limit root to change project state by default
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 8fad70c0872ba13133024e4abf53a0bbee7ba1e9
Landed for 2.14. sthiell yes it's flagged to be back ported!
Wang Shilong (wshilong@ddn.com) uploaded a new patch: https://review.whamcloud.com/37056
Subject: LU-12826 mdt: limit root to change project state by default
Project: fs/lustre-release
Branch: b2_12
Current Patch Set: 1
Commit: 993b9274805056998282eaac28523841ceaa26ff
Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/37056/
Subject: LU-12826 mdt: limit root to change project state by default
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: 11c9ec6384070c0627b45b17e16e1a1c9aea7249