[LU-12056] tar doesn't support project id Created: 10/Mar/19  Updated: 05/Dec/22  Resolved: 06/Jan/22

Status: Resolved
Project: Lustre
Component/s: None
Affects Version/s: None
Fix Version/s: Lustre 2.15.0

Type: Bug Priority: Critical
Reporter: Shuichi Ihara Assignee: Dongyang Li
Resolution: Fixed Votes: 0
Labels: ldiskfs
Environment:

master


Issue Links:
Related
is related to LU-4017 Add project quota support feature Resolved
is related to LU-11872 Request for option not to follow syml... Resolved
is related to LU-12480 add STATX_PROJID to upstream kernel Open
is related to LU-13335 add name lookup for project IDs Open
is related to LU-15372 add projid to changelog Open
Severity: 3
Rank (Obsolete): 9223372036854775807

 Description   

tar is not able to keep project id and it won't work for file backup/restart with project id.
This is not only lustre problem, but also same problem in ldiskfs. If MDT needs to be migration from old MDT to new MDT, tar discards project id.
There was discussion on LU-4017, I think we need to address this problem.



 Comments   
Comment by Andreas Dilger [ 10/Mar/19 ]

Probably it makes sense for Shilong to raise this issue on linux-fsdevel and bug-tar mailing lists to see if there is a preferred solution to saving/restoring the projid. I think that since the ioctl() interface is existingfor all filesystems using projid that it makes sense for tar to use this interface, but it would mean we need to patch tar until it is accepted into upstream versions/distros.

Providing an xattr to get/set the projid would be more compatible with other tools (even symlinks, I think), but it would mean changes to all filesystems using projid.

Comment by Wang Shilong (Inactive) [ 11/Mar/19 ]

Currently, I think solution might be to use get ioctl to grab project of directories and files and to use
set ioctl to set it, this won't work for symlink files until project ID for symlink files support landed.

I would think it makes more sense to add complete support to tar after symlink files support land too.

Comment by Andreas Dilger [ 21/Nov/20 ]

I think it would be relatively straight forward to expose a virtual xattr trusted.projid that returns only the ASCII-formatted projid value from i_projid in the inode, or nothing if it is zero. When written, it will convert the ASCII number to binary and store it in i_projid in the inode, similar to what FS_IOC_FSGETXATTR does.

This would immediately work with all backup tools, and would not need much code in ldiskfs, and would likely be accepted upstream as well.

It may also make sense to expose the inode flags via trusted.attrflags so that they are reliably backed up and restored as well.

Comment by Wang Shilong (Inactive) [ 24/Nov/20 ]

Yup, acutally, i've made this similar idea patch long time ago, but not push it out.

Comment by Andreas Dilger [ 21/Nov/21 ]

Shilong, you mentioned here that you have a patch that presents the projid as a virtual xattr to userspace. Do you still have that patch?

Comment by Andreas Dilger [ 23/Nov/21 ]

Hi Dongyang,
could you please look into an ldiskfs patch to expose the projid as a virtual trusted.projid that contains the ASCII project ID, so that normal tools like tar and rsync can backup and restore it. The use of trusted should limit this access to the root user, which is fine for us, and avoids the issue of non-root users using setfattr to set the projid on a file.

If ldiskfs is exposing the trusted.projid xattr in the xattr list, then this should also be visible to root at the client without any need for changes.

Comment by Gerrit Updater [ 30/Nov/21 ]

"Li Dongyang <dongyangli@ddn.com>" uploaded a new patch: https://review.whamcloud.com/45679
Subject: LU-12056 ldiskfs: add trusted.projid virtual xattr
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 34a9711a083a0d8a311b7caf2aaf444568eb6f83

Comment by Gerrit Updater [ 06/Jan/22 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/45679/
Subject: LU-12056 ldiskfs: add trusted.projid virtual xattr
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 665383d3a1f4d1dc7f404301039432271ad85eaf

Comment by Peter Jones [ 06/Jan/22 ]

Landed for 2.15

Comment by Gerrit Updater [ 25/Jan/22 ]

"Li Dongyang <dongyangli@ddn.com>" uploaded a new patch: https://review.whamcloud.com/46295
Subject: LU-12056 ldiskfs: add ext4-projid-xattrs.patch for Linux 5.10
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4086dcc2cd0757b3738cf32972020c6a54f1486d

Comment by Gerrit Updater [ 31/Jan/22 ]

"Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/46295/
Subject: LU-12056 ldiskfs: add ext4-projid-xattrs.patch for Linux 5.10
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 46a28ae19f38f8959986dcce2b331ac7e48886fc

Generated at Sat Feb 10 02:49:17 UTC 2024 using Jira 9.4.14#940014-sha1:734e6822bbf0d45eff9af51f82432957f73aa32c.