Details

    • Bug
    • Resolution: Fixed
    • Critical
    • Lustre 2.15.0
    • None
    • master
    • 3
    • 9223372036854775807

    Description

      tar is not able to keep project id and it won't work for file backup/restart with project id.
      This is not only lustre problem, but also same problem in ldiskfs. If MDT needs to be migration from old MDT to new MDT, tar discards project id.
      There was discussion on LU-4017, I think we need to address this problem.

      Attachments

        Issue Links

          Activity

            [LU-12056] tar doesn't support project id

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/46295/
            Subject: LU-12056 ldiskfs: add ext4-projid-xattrs.patch for Linux 5.10
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 46a28ae19f38f8959986dcce2b331ac7e48886fc

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/46295/ Subject: LU-12056 ldiskfs: add ext4-projid-xattrs.patch for Linux 5.10 Project: fs/lustre-release Branch: master Current Patch Set: Commit: 46a28ae19f38f8959986dcce2b331ac7e48886fc

            "Li Dongyang <dongyangli@ddn.com>" uploaded a new patch: https://review.whamcloud.com/46295
            Subject: LU-12056 ldiskfs: add ext4-projid-xattrs.patch for Linux 5.10
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 4086dcc2cd0757b3738cf32972020c6a54f1486d

            gerrit Gerrit Updater added a comment - "Li Dongyang <dongyangli@ddn.com>" uploaded a new patch: https://review.whamcloud.com/46295 Subject: LU-12056 ldiskfs: add ext4-projid-xattrs.patch for Linux 5.10 Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 4086dcc2cd0757b3738cf32972020c6a54f1486d
            pjones Peter Jones added a comment -

            Landed for 2.15

            pjones Peter Jones added a comment - Landed for 2.15

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/45679/
            Subject: LU-12056 ldiskfs: add trusted.projid virtual xattr
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 665383d3a1f4d1dc7f404301039432271ad85eaf

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/45679/ Subject: LU-12056 ldiskfs: add trusted.projid virtual xattr Project: fs/lustre-release Branch: master Current Patch Set: Commit: 665383d3a1f4d1dc7f404301039432271ad85eaf

            "Li Dongyang <dongyangli@ddn.com>" uploaded a new patch: https://review.whamcloud.com/45679
            Subject: LU-12056 ldiskfs: add trusted.projid virtual xattr
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 34a9711a083a0d8a311b7caf2aaf444568eb6f83

            gerrit Gerrit Updater added a comment - "Li Dongyang <dongyangli@ddn.com>" uploaded a new patch: https://review.whamcloud.com/45679 Subject: LU-12056 ldiskfs: add trusted.projid virtual xattr Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 34a9711a083a0d8a311b7caf2aaf444568eb6f83

            Hi Dongyang,
            could you please look into an ldiskfs patch to expose the projid as a virtual trusted.projid that contains the ASCII project ID, so that normal tools like tar and rsync can backup and restore it. The use of trusted should limit this access to the root user, which is fine for us, and avoids the issue of non-root users using setfattr to set the projid on a file.

            If ldiskfs is exposing the trusted.projid xattr in the xattr list, then this should also be visible to root at the client without any need for changes.

            adilger Andreas Dilger added a comment - Hi Dongyang, could you please look into an ldiskfs patch to expose the projid as a virtual trusted.projid that contains the ASCII project ID, so that normal tools like tar and rsync can backup and restore it. The use of trusted should limit this access to the root user, which is fine for us, and avoids the issue of non-root users using setfattr to set the projid on a file. If ldiskfs is exposing the trusted.projid xattr in the xattr list, then this should also be visible to root at the client without any need for changes.

            Shilong, you mentioned here that you have a patch that presents the projid as a virtual xattr to userspace. Do you still have that patch?

            adilger Andreas Dilger added a comment - Shilong, you mentioned here that you have a patch that presents the projid as a virtual xattr to userspace. Do you still have that patch?

            Yup, acutally, i've made this similar idea patch long time ago, but not push it out.

            wshilong Wang Shilong (Inactive) added a comment - Yup, acutally, i've made this similar idea patch long time ago, but not push it out.

            I think it would be relatively straight forward to expose a virtual xattr trusted.projid that returns only the ASCII-formatted projid value from i_projid in the inode, or nothing if it is zero. When written, it will convert the ASCII number to binary and store it in i_projid in the inode, similar to what FS_IOC_FSGETXATTR does.

            This would immediately work with all backup tools, and would not need much code in ldiskfs, and would likely be accepted upstream as well.

            It may also make sense to expose the inode flags via trusted.attrflags so that they are reliably backed up and restored as well.

            adilger Andreas Dilger added a comment - I think it would be relatively straight forward to expose a virtual xattr trusted.projid that returns only the ASCII-formatted projid value from i_projid in the inode, or nothing if it is zero. When written, it will convert the ASCII number to binary and store it in i_projid in the inode, similar to what FS_IOC_FSGETXATTR does. This would immediately work with all backup tools, and would not need much code in ldiskfs, and would likely be accepted upstream as well. It may also make sense to expose the inode flags via trusted.attrflags so that they are reliably backed up and restored as well.

            Currently, I think solution might be to use get ioctl to grab project of directories and files and to use
            set ioctl to set it, this won't work for symlink files until project ID for symlink files support landed.

            I would think it makes more sense to add complete support to tar after symlink files support land too.

            wshilong Wang Shilong (Inactive) added a comment - Currently, I think solution might be to use get ioctl to grab project of directories and files and to use set ioctl to set it, this won't work for symlink files until project ID for symlink files support landed. I would think it makes more sense to add complete support to tar after symlink files support land too.

            People

              dongyang Dongyang Li
              sihara Shuichi Ihara
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: