https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 https://jira.whamcloud.com/browse/LU-11872 Lustre <p>Currently, when issuing "lfs project -srp xxxx DIR", command follows symbolic link and set project ID for the target of the link. Customer want a new option not to follow symbolic link.</p> <p>Can you please think on implementing a new option?</p> <p>They want to have quota based on directory.<br/> For instance, when there are two users and symlinks are created as follows,</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> /home/user01/backup/link01 -&gt; /home/user02/backup/file01 /home/user02/backup/link02 -&gt; /home/user01/backup/file02 </pre> </div></div> <p>Then, when they issue 'lfs project' command,</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> lfs project -p 10001 -s -r /home/user01/backup lfs project -p 10002 -s -r /home/user02/backup </pre> </div></div> <p>Project ID for /home/user01/backup/file02 will be "10002". As they want project IDs for all the files under /home/user01/backup to be "10001", considering this kind of possibility, they need to check all the files under the directory. They just want to simply run a command on a directory. </p> lustre 2.7.21.3-ddn33 LU-11872

Request for option not to follow symlink when setting project ID

New Feature Minor Resolved Fixed Wang Shilong Mitsuhiro Nishizawa Fri, 18 Jan 2019 06:14:06 +0000 Tue, 16 May 2023 14:50:41 +0000 Tue, 13 Jul 2021 10:04:04 +0000 Lustre 2.15.0 0 14 <p><a href="https://jira.whamcloud.com/secure/ViewProfile.jspa?name=wshilong" class="user-hover" rel="wshilong">wshilong</a> could you please advise as to how much effort it would be to add such a feature in a future version of Lustre?</p> <p>It should be fairly low effort to just skip symlinks when setting the project ID by default. Setting the project ID on the symlinks themselves may be more complex and could be deferred to e2fsck if they are problematic.</p> <p>Andreas,</p> <p> I think problem is we should still be able to change project ID of symlink file as chown could.<br/> Current problem is there is no hook function to handle this in the kernel.</p> <p>One of simple solution could be we passing parent dir, and symlink inode number in @fsxattr.<br/> and if child inode number is passed in, ioctl will affect to child rather than parents.</p> <p>What do you think this idea?</p> <p>I think this should be split into two separate patches.</p> <p>The first patch is for skipping symlinks, since this is easily done and can be backported with little effort since it only affects "lfs". Then an RFC patch for "setfattr" in e2fsprogs, to start a discussion on how this should be fixed properly, with a CC to linux-fsdevel to ask how XFS has solved this problem. </p> <p>Finally, a second patch for Lustre to set the project ID for symlinks can be implemented once we have some idea how it should be done.</p> <p>Hi Andreas,</p> <p> Fair enough, btw, I think XFS did not solve this problem too..<br/> Let's do this anyway.</p> <p>Wang Shilong (wshilong@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/34111" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34111</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11872" title="Request for option not to follow symlink when setting project ID" class="issue-link" data-issue-key="LU-11872"><del>LU-11872</del></a> utils: don't follow link files in default<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 07ff9a3a5a1f333b7e14a90ce991b4f1eea80f57</p> <p>There are some discussions flighting on:</p> <p>1. One way is to recreate symlink which will inherit projectID from parents automatically,<br/> This way works to implement directory Quota, but it doesn't work to set ProjectID of symlink<br/> to a &lt;ID&gt; different from parents, and list symfile's project still not supported.</p> <p>2. Upstream kernel suggest to add a new flag AT_PROJID and then use fchown_at() to indicate uid is actually projid.</p> <p>More questions for upstream suggestion:</p> <div class="preformatted panel" style="border-width: 1px;"><div class="preformattedContent panelContent"> <pre>For Project Set: We might need introduce two extra flags here, one flag indicate use group id to set Project ID. Another flag means set project Inherit attribute using fchown_at(). How about Get Project? It looks like there is no extra var for projectID in "struct ksta{", we could use @attributes to return inherit set or not. </pre> </div></div> <p>Some updates:</p> <p>Send a generic interface to Linux Kernel community for long term goal.<br/> <a href="http://patchwork.ozlabs.org/cover/1050205/" class="external-link" target="_blank" rel="nofollow noopener">http://patchwork.ozlabs.org/cover/1050205/</a></p> <p>Wang Shilong (wshilong@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/34447" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34447</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11872" title="Request for option not to follow symlink when setting project ID" class="issue-link" data-issue-key="LU-11872"><del>LU-11872</del></a> utils: add option to recreate symlink files<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: ed22649195111890d81f258054f17f2b19e8bdcd</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/34111/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/34111/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11872" title="Request for option not to follow symlink when setting project ID" class="issue-link" data-issue-key="LU-11872"><del>LU-11872</del></a> utils: don't follow link files in default<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: 004b80da5c4b2a7cf4f4885b43c9edec76cd2493</p> <p>Wang Shilong (wshilong@ddn.com) uploaded a new patch: <a href="https://review.whamcloud.com/35392" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35392</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11872" title="Request for option not to follow symlink when setting project ID" class="issue-link" data-issue-key="LU-11872"><del>LU-11872</del></a> utils: don't follow link files in default<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: 1<br/> Commit: a326d5792151d1bd956c820ea1c689154a984ec6</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/35392/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/35392/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11872" title="Request for option not to follow symlink when setting project ID" class="issue-link" data-issue-key="LU-11872"><del>LU-11872</del></a> utils: don't follow link files in default<br/> Project: fs/lustre-release<br/> Branch: b2_12<br/> Current Patch Set: <br/> Commit: 4cb79268abe8c9405195436e8e8e2ee16577694d</p> <p>Edit: realized this should probably be a new issue, as this one is tailored to symlinks. There is similar behavior with hardlinks as well as some additional odd behavior.</p> <p>Hardlink issue: <a href="https://jira.whamcloud.com/browse/LU-14062" title="Setting project ID inheritance on directories can break hardlinking" class="issue-link" data-issue-key="LU-14062">LU-14062</a></p> <p>Can the fix to "recreate symlink files" be moved forward?</p> <p>The incorrect project ID is preventing migration of directories to new MDTs (DNE1).  Although, this may be revealing a different bug in the "lfs migrate" capability for MDTs getting stuck when project IDs are not fully matching (as hinted at in <a href="https://jira.whamcloud.com/browse/LU-14062" title="Setting project ID inheritance on directories can break hardlinking" class="issue-link" data-issue-key="LU-14062">LU-14062</a>).  Here is a test case:</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> # lfs project -d .   0 - . # mkdir mdt_migration_test # ln -s .. mdt_migration_test/symlink # lfs project -p 999 -r -s mdt_migration_test lfs: unable to get xattr <span class="code-keyword">for</span> symlink <span class="code-quote">'mdt_migration_test/symlink'</span>: Operation not supported # lfs migrate -m 3 mdt_migration_test mdt_migration_test/symlink migrate failed: Invalid cross-device link (-18) # lfs getdirstripe mdt_migration_test lmv_stripe_count: 2 lmv_stripe_offset: 3 lmv_hash_type: fnv_1a_64,migrating mdtidx FID[seq:oid:ver]   3 [0x24000ee61:0x3:0x0]   0 [0x200022085:0x8:0x0] </pre> </div></div> <p>The only way I have found to get out of the "migrating" state is to clear the project information (so the symlink matches the parent again), redo the migation, and then re-change the project ID. Of course this is quick on a small test case, but for a real directory it could take a <b>long</b> time to walk the tree and change back and forth and the project usage information would be incorrect in the meantime.</p> <div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent"> <pre class="code-java"> # lfs project -C -r mdt_migration_test lfs: unable to get xattr <span class="code-keyword">for</span> symlink <span class="code-quote">'mdt_migration_test/symlink'</span>: Operation not supported # lfs getdirstripe mdt_migration_test lmv_stripe_count: 2 lmv_stripe_offset: 3 lmv_hash_type: fnv_1a_64,migrating mdtidx FID[seq:oid:ver] 3 [0x24000ee61:0x3:0x0] 0 [0x200022085:0x8:0x0] # lfs project -d mdt_migration_test 0 - mdt_migration_test # lfs migrate -m 3 mdt_migration_test # lfs getdirstripe mdt_migration_test lmv_stripe_count: 0 lmv_stripe_offset: 3 lmv_hash_type: none # lfs project -p 999 -r -s mdt_migration_test lfs: unable to get xattr <span class="code-keyword">for</span> symlink <span class="code-quote">'mdt_migration_test/symlink'</span>: Operation not supported </pre> </div></div> <p>I will refresh patch.</p> <blockquote> <p>It should be fairly low effort to just skip symlinks when setting the project ID by default. Setting the project ID on the symlinks themselves may be more complex and could be deferred to e2fsck if they are problematic.</p></blockquote> <p>Andreas, are you about not setting/inheriting project id for symlinks while creation? it would be the simplest approach. Inheriting project id from parent while creating a symlink and not having a way to change it later looks not right.</p> <p>Alexander, since symlinks consume inodes, they need to be accounted in the quota, otherwise admins will complain that "find" and the quota count do not match. I think the 34447 patch provides a reasonable solution to the (AFAIK) uncommon case of changing the projid of symlink files, until such a time that the kernel allows this to be done directly. </p> <p>Alexander Zarochentsev (alexander.zarochentsev@hpe.com) uploaded a new patch: <a href="https://review.whamcloud.com/43965" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/43965</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11872" title="Request for option not to follow symlink when setting project ID" class="issue-link" data-issue-key="LU-11872"><del>LU-11872</del></a> ldiskfs: do not inherit project id for links<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: 5f140302f9327afd803149e0081df8068eb7fd2e</p> <p>Alexander, any comment on the resulting inconsistency between project quota inode usage and what "find" would report for the directory, or even that user and group quota will report for the same directory tree? </p> <p>I don't think it good idea to "do not inherit project id for links", we tried to introduce a new ioctl for Lustre which passed</p> <p>Parent dir and child name to get/set project ID, this is even better idea than this IMO..</p> <p> </p> <p>&gt;Alexander, any comment on the resulting inconsistency between project quota inode usage and what "find" would report for the directory, or even that user and group quota will report for the same directory tree?</p> <p>yes it has a drawback. I want to get a feedback whether it is a serious problem or not. <a href="https://review.whamcloud.com/#/c/34447" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/34447</a> is not updated for about a year, are there plans to move it forward?</p> <p>&gt; we tried to introduce a new ioctl for Lustre which passed<br/> can <a href="https://review.whamcloud.com/#/c/34447" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/34447</a> be accepted as a temporary solution till the new kernel interface for changing project id is added to the vanilla kernel and that new kernel gets used by RH?</p> <p>I recalled when last time i tried to refresh, i was aware of that patch also has a problem even we could recreate symlinks, because there might still some other type of files could have similar problems eg fifo, so it did not fix problem completely though.</p> <p> </p> <p>I could move forward the patch(sorry for delay), even though i prefer idea new ioctl(parent dir and child name), because that could also fix problem get project ID of symlinks/fifo..</p> <p> </p> <p>Any though?</p> <p> </p> <p>It might take some time to have vanilla kernel to support get/change projectID, i pushed a series patch to upstream to change projectID, and we might extend statx() to get project ID of files, but that might take few years to be RH kernel, so i was still wondering if it ok to a walkaround ioctl in Lustre before that is better to fix problem completely.. rather than continue <a href="https://review.whamcloud.com/#/c/34447" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/34447</a> patch? </p> <p>&gt;if it ok to a walkaround ioctl in Lustre before that is better to fix problem completely.. rather than continue <a href="https://review.whamcloud.com/#/c/34447" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/#/c/34447</a> patch.<br/> I think yes.</p> <p>Wang Shilong (wshilong@whamcloud.com) uploaded a new patch: <a href="https://review.whamcloud.com/44006" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44006</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11872" title="Request for option not to follow symlink when setting project ID" class="issue-link" data-issue-key="LU-11872"><del>LU-11872</del></a> quota: add get/set project support for non-dir/file<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: 1<br/> Commit: b34dff90fc6f8d91640f68ee4cf7acfb70bebe17</p> <p>Oleg Drokin (green@whamcloud.com) merged in patch <a href="https://review.whamcloud.com/44006/" class="external-link" target="_blank" rel="nofollow noopener">https://review.whamcloud.com/44006/</a><br/> Subject: <a href="https://jira.whamcloud.com/browse/LU-11872" title="Request for option not to follow symlink when setting project ID" class="issue-link" data-issue-key="LU-11872"><del>LU-11872</del></a> quota: add get/set project support for non-dir/file<br/> Project: fs/lustre-release<br/> Branch: master<br/> Current Patch Set: <br/> Commit: b31792b0e72425c8c7850d69837f08c9f3e95a9c</p> <p>Patch landed</p> <p>A possibly better solution instead of adding a specific ioctl for this is to use <tt>lgetxattr()</tt> and <tt>lsetxattr()</tt> to get/set the <tt>trusted.projid</tt> virtual xattr from <a href="https://jira.whamcloud.com/browse/LU-12056" title="tar doesn&#39;t support project id" class="issue-link" data-issue-key="LU-12056"><del>LU-12056</del></a> on symlinks. That will also work for other utilities beyond "<tt>lfs project</tt>" and also works on ldiskfs-mounted MDTs/OSTs directly.</p> Duplicate Related LU-12480 LU-14062 LU-12056 LU-10501 LU-12947 LU-16808 LU-11631 Development Rank 1|i009vz: Rank (Obsolete) 9223372036854775807