https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-12826 Lustre <p>We're considering using project quotas in the future but we noticed that users can change or clear the project id of their own files/directories. How can we restrict access to project IDs only to admins (like ext4 does I guess?). We looked at the manual but couldn't find how permissions are handled. Sorry if we missed something obvious.</p> <p> </p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[root@fir-io7-s2 users]# ls -ld /firhdr/users/sthiell drwxr-x--- 4 sthiell root 4096 Sep 30 20:30 /firhdr/users/sthiell [root@fir-io7-s2 users]# lfs project -d /firhdr/users/sthiell 10 P /firhdr/users/sthiell [root@fir-io7-s2 users]# su -m sthiell bash-4.2$ cd /firhdr/users/sthiell bash-4.2$ lfs project -C . bash-4.2$ lfs project -d . 0 - . </pre> </div></div> CentOS 7.6, ldiskfs, 2.12.2_116+3 LU-12826

Project quotas: users can change project IDs

Bug Major Resolved Fixed Wang Shilong Stephane Thiell Tue, 1 Oct 2019 04:25:48 +0000 Fri, 3 Jan 2020 23:58:23 +0000 Sat, 14 Dec 2019 13:39:13 +0000 Lustre 2.12.2 Lustre 2.14.0 Lustre 2.12.4 0 11 <p>Shilong</p> <p>Could you please advise?</p> <p>Thanks</p> <p>Peter</p> <p>This could be a little tricky, currently Lustre did what ext4/xfs does:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>int ll_ioctl_check_project(struct inode *inode, struct fsxattr *fa) { /* * Project Quota ID state is only allowed to change from within the init * namespace. Enforce that restriction only if we are trying to change * the quota ID state. Everything else is allowed in user namespaces. */ if (current_user_ns() == &amp;init_user_ns) return 0; </pre> </div></div> <p>Since you are switching to sthiell from root, it is in still in same namespace, i guess you could<br/> try ssh sthiell@fir-io7-s2 and then do same operations which it will return no permissions errors.</p> <p>Hi Shilong,</p> <p>Thanks for your reply! This sounded a good point at first, so I tried without su, with ssh sthiell@fir-io7-s2 but same thing: I was able to modify and even clear the project ID as a user. You said it will return "no permissions errors", actually we want that it returns an error in that case (something like permission denied). Did we miss anything else?</p> <p>Yup, you are right.</p> <p>The point is it looks current check is broken somehow.</p> <p>Following check always return true in Centos7 client:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>current_user_ns() == &amp;init_user_ns </pre> </div></div> <p>Will check more with this.</p> <p>This is a bit interesting, currently XFS/ext4 did same thing for this, that owner could change project ID of files, following is testing results of XFS:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>[wangsl@server wangsl]$ chattr -p 1 foo [wangsl@server wangsl]$ lsattr -p foo 1 ------------------- foo [wangsl@server wangsl]$ chattr -p 2 foo [wangsl@server wangsl]$ lsattr -p foo 2 ------------------- foo [wangsl@server wangsl]$ df . -Th Filesystem Type Size Used Avail Use% Mounted on /dev/sda4 xfs 48G 82M 48G 1% /mnt [wangsl@server wangsl]$ uname -r 5.1.17-300.fc30.x86_64 </pre> </div></div> <p>This is a bit different behavior than originally we think for lustre project quota use case, we could<br/> add ROOT CAP check for lustre for surely.</p> <p>Hi Shilong, </p> <p>Thanks for checking! It's a bit of a disappointment,, as projects quotas don't seem to be usable as-is in a shared environment context. If users can easily circumvent project quotas by simply changing the project id on their files, there's not much point in enforcing project quotas in the first place.</p> <p>Now, for accounting purposes, there probably are valid use cases to allow users to set project ids on specific directories, and avoid having to rely on `du` to get the total volume/inode usage of a given file hierarchy for instance, project quotas would give that information right away.</p> <p>So maybe there's a need for a more flexible tunable? To make changing project ids on files and directory either user-accessible, or a root-only action, depending on the use case?</p> <p>To be honest, I'm a bit stumped by all of this: how are people using project quotas in practice (not only with Lustre, but with XFS or ext4 as well) if regular users can switch project ids as they wish?</p> <p>In any case, thanks a lot for looking into it!</p> <p>Cheers,<br/> -- <br/> Kilian</p> <p> </p> <p>It probably makes sense to raise this as a question on the linux-fsdevel and/or linux-ext4 mailing lists. It is probably best to not mention Lustre and instead focus on the behavior of ext4 and/or xfs on the lists. </p> <p>Yup, let me send an email to upstream to raise this problem.</p> <p>I just raised this problem in the upstream linux mail list:</p> <p><a href="https://marc.info/?l=linux-xfs&amp;m=157086204800850&amp;w=2" class="external-link" target="_blank" rel="nofollow noopener">https://marc.info/?l=linux-xfs&amp;m=157086204800850&amp;w=2</a></p> <p>After understanding the reasons of the project ID behavior in the mail list, I still think the current behavior (not allowing the file owner to change the project ID of its file) is still useful in a lot of use cases. It would be a lose to change the current Lustre behavior to the same with Ext4/XFS, especially considering that most likely Lustre are used in the circumstances with more users than local file systems. Another factor is Lustre usually have much larger size than local file systems. That means if any user walks around the project quota limitations, it could troubles to the administrators. Thus, I don't think it is proper to change the current bahvior of Lustre project quota .</p> <p>Nevertheless, it is still valuable to enable some use cases of project quota similar to Ext4/XFS, especailly when containers are being used. Thus, I think it might be a good idea to add a new mount option to Lustre client like "prjid_owner_changable". If that option is enabled whem mounting, then it will have the same behavior like Ext4/XFS.</p> <p>In this way, I think we are enabling all kinds of use cases depending on the requirement. It is a question whether "prjid_owner_changable" should be enabled by default or not. I feel the answer is "no" at least for now.</p> <p>Let me push a patch to limit root for project change in default, and with prjid_owner_changeable option, users could change their project ID and inherit flags.</p> <p>I think rather than a mount option, it would be better to have a tunable on the MDS to control this, maybe along with a nodemap option as Sébastien has done in patch <a href="https://review.whamcloud.com/36433" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36433</a> so that it can be controlled by the server instead of the client, which may be a VM under the control of an unfriendly user. </p> <p>PS: I agree that regular users should not be able to change the projid by default. </p> <p>and I agree with Andreas <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>&gt; rather than a mount option, it would be better to have a tunable on the MDS to control this</p> <p>Agreed on that.</p> <p>Hi Li Xi,</p> <p>Just to clarify, when you say:</p> <p>&gt; I still think the current behavior (not allowing the file owner to change the project ID of its file) is still useful in a lot of use cases. </p> <p>It is actually the opposite: currently in Lustre the file owner can change the project ID of its file. That means on filesystems where project quotas are enforced, users can likely bypass their project quota if they use <tt>lfs project</tt> or <tt>chattr -p</tt> (if available on the cluster) on their own files. We need a fix to only allow admins to set/change/remove project IDs and this should be the default, as in my understanding, the most common use case with Lustre is to use project quotas to enforce some kind of directory quotas (with inheritance), managed by system administrators, not users.</p> <p>But a tunable on the server should solve that. <img class="emoticon" src="https://jira.whamcloud.com/images/icons/emoticons/smile.png" height="16" width="16" align="absmiddle" alt="" border="0"/></p> <p>I think it makes sense to add a tunable on the MDS similar to "<tt>mdt.&#42;.enable&#95;remote&#95;dir&#95;gid</tt>" that allows users in specific groups to also change the projid. By default, I think it makes sense to default to only allow root (<tt>CAP&#95;SYS&#95;RESOURCE</tt>) to change the projid directly, and set "<tt>mdt.&#42;.change&#95;projid&#95;gid=0</tt>". Any users in that group (e.g. <tt>wheel</tt> or <tt>admin</tt>) can also change projid directly, if it is "<tt>-1</tt>" then any user can change it.</p> <p>Wang Shilong (wshilong@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/36544" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36544</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12826" title="Project quotas: users can change project IDs" class="issue-link" data-issue-key="LU-12826"><del>LU-12826</del></a> mdt: limit root to change project state<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 3d38465be14aa28b89f75384980c4e96e15a1f04</p> <p>It would be nice to have a backport of this important patch to b2_12 when it has landed (I tried to apply it but it doesn't work as is). Thanks much!</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/36544/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/36544/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12826" title="Project quotas: users can change project IDs" class="issue-link" data-issue-key="LU-12826"><del>LU-12826</del></a> mdt: limit root to change project state by default<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 8fad70c0872ba13133024e4abf53a0bbee7ba1e9</p> <p>Landed for 2.14. <a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=sthiell" class="user-hover" rel="sthiell">sthiell</a> yes it's flagged to be back ported!</p> <p>Wang Shilong (wshilong@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/37056" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37056</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12826" title="Project quotas: users can change project IDs" class="issue-link" data-issue-key="LU-12826"><del>LU-12826</del></a> mdt: limit root to change project state by default<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: 993b9274805056998282eaac28523841ceaa26ff</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/37056/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/37056/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-12826" title="Project quotas: users can change project IDs" class="issue-link" data-issue-key="LU-12826"><del>LU-12826</del></a> mdt: limit root to change project state by default<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: 11c9ec6384070c0627b45b17e16e1a1c9aea7249</p> Related LU-11303 Development Rank 1|i00nkv: Rank (Obsolete) 9223372036854775807 Severity