Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10070

PFL self-extending file layout

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: Lustre 2.10.0
    • Fix Version/s: Lustre 2.13.0
    • Labels:
    • Rank (Obsolete):
      9223372036854775807

      Description

      One interesting idea discussed at LAD was to have a PFL layout that is "self extending".

      For several use cases, such as HSM partial-file release/restore, partial-file migration to/from burst buffers, partial-file FLR resync it is advantageous to avoid the need to restore/migrate/resync the entire file at once, but rather only to process the required chunks of the file.

      Essentially, a PFL file would have the normal few components that define the start of the file (e.g. [0-32MB), [32MB-1GB), [1GB-16GB)) and they would be instantiated as with normal PFL today. What is new for "self-extending layouts" is that the last component becomes the template for additional components (as needed) rather than having a component to "EOF" that freezes the layout of the rest of the file.

      This avoids the overhead of explicitly specifying many identical components for the file, only in order to limit the size of the components that need to be processed.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vitaly_fertman Vitaly Fertman
                Reporter:
                adilger Andreas Dilger
              • Votes:
                0 Vote for this issue
                Watchers:
                24 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: