Details

    • New Feature
    • Resolution: Fixed
    • Minor
    • Lustre 2.13.0
    • Lustre 2.10.0
    • 9223372036854775807

    Description

      One interesting idea discussed at LAD was to have a PFL layout that is "self extending".

      For several use cases, such as HSM partial-file release/restore, partial-file migration to/from burst buffers, partial-file FLR resync it is advantageous to avoid the need to restore/migrate/resync the entire file at once, but rather only to process the required chunks of the file.

      Essentially, a PFL file would have the normal few components that define the start of the file (e.g. [0-32MB), [32MB-1GB), [1GB-16GB)) and they would be instantiated as with normal PFL today. What is new for "self-extending layouts" is that the last component becomes the template for additional components (as needed) rather than having a component to "EOF" that freezes the layout of the rest of the file.

      This avoids the overhead of explicitly specifying many identical components for the file, only in order to limit the size of the components that need to be processed.

      Attachments

        Issue Links

          Activity

            [LU-10070] PFL self-extending file layout
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-9096 [ LU-9096 ]
            pjones Peter Jones made changes -
            Link New: This issue is related to EX-5612 [ EX-5612 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to EX-3295 [ EX-3295 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-13589 [ LU-13589 ]
            mdiep Minh Diep made changes -
            Link New: This issue is related to EX-1004 [ EX-1004 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-13058 [ LU-13058 ]
            adilger Andreas Dilger made changes -
            Link New: This issue is related to LU-13395 [ LU-13395 ]

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36554/
            Subject: LU-10070 utils: move new SEL find_param fields to end
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 5222ba07e3bd087ddb0812e2185610b725cb9d1a

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/36554/ Subject: LU-10070 utils: move new SEL find_param fields to end Project: fs/lustre-release Branch: master Current Patch Set: Commit: 5222ba07e3bd087ddb0812e2185610b725cb9d1a

            Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36554
            Subject: LU-10070 utils: move new SEL find_param fields to end
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: ba7fe211563652856c481a694d0fdaddad3d8dfe

            gerrit Gerrit Updater added a comment - Andreas Dilger (adilger@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/36554 Subject: LU-10070 utils: move new SEL find_param fields to end Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: ba7fe211563652856c481a694d0fdaddad3d8dfe
            pjones Peter Jones made changes -
            Resolution New: Fixed [ 1 ]
            Status Original: Open [ 1 ] New: Resolved [ 5 ]

            People

              vitaly_fertman Vitaly Fertman
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: