Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-10070

PFL self-extending file layout

    XMLWordPrintable

Details

    • New Feature
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • Lustre 2.10.0
    • Lustre 2.13.0
    • 9223372036854775807

    Description

      One interesting idea discussed at LAD was to have a PFL layout that is "self extending".

      For several use cases, such as HSM partial-file release/restore, partial-file migration to/from burst buffers, partial-file FLR resync it is advantageous to avoid the need to restore/migrate/resync the entire file at once, but rather only to process the required chunks of the file.

      Essentially, a PFL file would have the normal few components that define the start of the file (e.g. [0-32MB), [32MB-1GB), [1GB-16GB)) and they would be instantiated as with normal PFL today. What is new for "self-extending layouts" is that the last component becomes the template for additional components (as needed) rather than having a component to "EOF" that freezes the layout of the rest of the file.

      This avoids the overhead of explicitly specifying many identical components for the file, only in order to limit the size of the components that need to be processed.

      Attachments

        Issue Links

          Activity

            People

              vitaly_fertman Vitaly Fertman
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: