[LU-13176] rename() to another directory should transfer project quota Created: 28/Jan/20  Updated: 04/Aug/23  Resolved: 31/Jan/22

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: Lustre 2.15.0

Type: Improvement Priority: Major
Reporter: Andreas Dilger Assignee: Hongchao Zhang
Resolution: Fixed Votes: 0
Labels: easy

Issue Links:
Duplicate
Related
is related to LU-11303 slow chgrp as user when quotas are en... Resolved
is related to LU-15667 Interop sanity-quota test_52: FAIL: r... Open
is related to LU-11303 slow chgrp as user when quotas are en... Resolved
is related to LU-15294 runtests: general protection fault: l... Resolved
is related to LU-17016 rename across projects incorrectly ch... Resolved
Rank (Obsolete): 9223372036854775807

 Description   

When rename() has a target directory with different project ID from the source directory, the kernel currently returns -EXDEV, which forces userspace to perform a full data copy of the file. This is slow when the files are large, and does not provide any security benefit for the user or admin in the end.

It would be better to change the project ID to that of the target directory, and assign the quota for the file to the new project ID, the same as is done with chown() and chgrp() for UID and GID quotas.

 Comments   
Comment by Andreas Dilger [ 28/Jan/20 ]

It looks like the reasoning behind this design choice is described in the email thread "quota: add project quota support" between the primary ext4 and XFS developers:

    > ... we should check with the XFS folks what happens if you move a file
    > from one directory which belongs to one project quota to another
    > directory that has a different project quota (or no quota whatsoever).

Rename to a destination with a different project quota gives EXDEV, same as if you were trying to rename across different filesystems. See xfs_rename().

    > I suspect the right answer is that the quota gets transferred from one
    > project to another, so that it is a true directory-tree quota system,

XFS doesn't transfer the quota from projid to projid because it's borderline impossible to correctly track all the metadata allocation/free operations that can happen in a rename operation and account them to the correct quota. Hence all those corner cases are avoided by treating it as EXDEV and forcing userspace to cp/unlink the files rather than rename.

I agree that this is much more of a problem for Lustre than it is for local filesystems. As a workaround for this behavior when moving a large directory tree, it seems possible to use "lfs project" to change the project ID of the source directory to match that of the target directory before the directory is moved, at which point the "mv" operation should just be doing a rename since the project IDs will be the same for the source and target.

Comment by Gerrit Updater [ 25/Nov/21 ]

"Hongchao Zhang <hongchao@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/45660
Subject: LU-13176 mdd: rename file with different project ID
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 2adb3f2bba3e693d1537917abd9038db5dc301f5

Comment by Gerrit Updater [ 31/Jan/22 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/45660/
Subject: LU-13176 mdd: rename file with different project ID
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 88c26912a3237fb63923bbb7c7b09111f9f30bbe

Comment by Peter Jones [ 31/Jan/22 ]

Landed for 2.15

Generated at Sat Feb 10 02:59:02 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.