Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-12714 PCC Phase 2
  3. LU-11829

Separate the naming structure between RW-PCC and RO-PCC

Details

    • Technical task
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • 9223372036854775807

    Description

      Currenly a PCC backend can provide caching services for both RW-PCC and RO-PCC which both use lhsmposix naming structure as follows:

       "%04x/%04x/%04x/%04x/%04x/%04x/" DFID_NOBRACE
      

      However, sharing the same naming structure may cause some confusions. For example, when RO-PCC cache a HSM archived file (not HSM released), it may confuse the user that whether the cached file object in the PCC device is RO-PCC cached file or a HSM archive. Moreover, a HSM remove request maybe delete the RO-PCC cached file wrongly.
      To solve this problem, we add a suffix .rdonly for RO-PCC naming strcuture to distingush from the RW-PCC and HSM solution:

      "%04x/%04x/%04x/%04x/%04x/%04x/"DFID_NOBRACE".rdonly"
      

      Attachments

        Activity

          [LU-11829] Separate the naming structure between RW-PCC and RO-PCC

          Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/38309
          Subject: LU-11829 pcc: Use different fnames for RW-PCC and RO-PCC modes
          Project: fs/lustre-release
          Branch: master
          Current Patch Set: 1
          Commit: dff3b6ab81c8e11cf79e4f67447fc884c7747aa5

          gerrit Gerrit Updater added a comment - Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/38309 Subject: LU-11829 pcc: Use different fnames for RW-PCC and RO-PCC modes Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: dff3b6ab81c8e11cf79e4f67447fc884c7747aa5

          Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/33942
          Subject: LU-11829 pcc: Separate the naming between RW-PCC and RO-PCC
          Project: fs/lustre-release
          Branch: pcc
          Current Patch Set: 1
          Commit: 4e0acfaf9d892db32da4fa99dc2665f8e28ac979

          gerrit Gerrit Updater added a comment - Yingjin Qian (qian@ddn.com) uploaded a new patch: https://review.whamcloud.com/33942 Subject: LU-11829 pcc: Separate the naming between RW-PCC and RO-PCC Project: fs/lustre-release Branch: pcc Current Patch Set: 1 Commit: 4e0acfaf9d892db32da4fa99dc2665f8e28ac979
          qian_wc Qian Yingjin added a comment -

          Hi Andreas,

          Thanks for you comments.

          I think we should create a new ticket to discuss and improve the usage efficiency of the current lhsmtool_posix naming convention for objects.

           

          Regards,

          Qian

          qian_wc Qian Yingjin added a comment - Hi Andreas, Thanks for you comments. I think we should create a new ticket to discuss and improve the usage efficiency of the current lhsmtool_posix naming convention for objects.   Regards, Qian

          Note also that the current lhsmtool_posix naming convention for objects is not very good, and should be avoided for new deployments. It currently puts the low part of the FID at the top of the directory tree, and the high part of the FID in the lower directories. This means that each directory leaf will only ever have a single filename in it until many billions of files are archived.

          Instead, the naming should be changed to do the opposite - put the high word of SEQ at the top of the directory tree, and the low word of the OID at the bottom. This will keep some reasonable number of objects in each leaf directory (eg. 65000) and keep a more compact directory tree.

          If this needs a change to the POSIX copytool, then for compatibility it should create a config file or marker at the top of the tree (eg.lhsm_posix_v2) and check for that at startup to determine if the new naming format should be used or the old one.

          adilger Andreas Dilger added a comment - Note also that the current lhsmtool_posix naming convention for objects is not very good, and should be avoided for new deployments. It currently puts the low part of the FID at the top of the directory tree, and the high part of the FID in the lower directories. This means that each directory leaf will only ever have a single filename in it until many billions of files are archived. Instead, the naming should be changed to do the opposite - put the high word of SEQ at the top of the directory tree, and the low word of the OID at the bottom. This will keep some reasonable number of objects in each leaf directory (eg. 65000) and keep a more compact directory tree. If this needs a change to the POSIX copytool, then for compatibility it should create a config file or marker at the top of the tree (eg. lhsm_posix_v2 ) and check for that at startup to determine if the new naming format should be used or the old one.

          People

            qian_wc Qian Yingjin
            qian_wc Qian Yingjin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: