Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-17159

Mark file layouts using append striping

Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • 3
    • 9223372036854775807

    Description

      While it is not necessary for functional reasons, it would be useful for debugging/diagnostic reasons to set a flag in the file layout when the MDS decides to use the append-specific layout. Otherwise, this can cause hard-to-diagnose behavior with file layouts when it overrides the default file layout.

      This could set the following flag in the layout when a file is using MDS_OPEN_APPEND and dah_append_stripe_count:

      enum lcm_comp_md_flags {
               :
               LCM_FL_INCOMPRESSIBLE   = 0x10,
      +        LCM_FL_APPEND           = 0x20,
      };
      

      The "lfs getstripe" and other commands would need to be updated for this flag.

      Attachments

        Issue Links

          Activity

            [LU-17159] Mark file layouts using append striping
            paf0186 Patrick Farrell added a comment -

            I have in theory corrected the thing making this happen.  We'll see.

            paf0186 Patrick Farrell added a comment - I have in theory corrected the thing making this happen.  We'll see.
            gerrit Gerrit Updater added a comment - - edited

            (wrong LU)

            gerrit Gerrit Updater added a comment - - edited (wrong LU)
            paf0186 Patrick Farrell added a comment -

            Still loving this LU, sorry

            paf0186 Patrick Farrell added a comment - Still loving this LU, sorry
            gerrit Gerrit Updater added a comment - - edited

            (wrong LU)

            gerrit Gerrit Updater added a comment - - edited (wrong LU)
            gerrit Gerrit Updater added a comment - - edited

            (wrong LU)

            gerrit Gerrit Updater added a comment - - edited (wrong LU)

            Ugh, not sure why Augment love this LU #.  Probably in the memories.

            paf0186 Patrick Farrell added a comment - Ugh, not sure why Augment love this LU #.  Probably in the memories.
            gerrit Gerrit Updater added a comment - - edited

            (incorrect LU)

            gerrit Gerrit Updater added a comment - - edited (incorrect LU)
            gerrit Gerrit Updater added a comment - - edited

            (incorrect LU)

            gerrit Gerrit Updater added a comment - - edited (incorrect LU)

            One issue that was raised at LUG with the append-only layouts is that they are confusing for users, since they don't follow normal per-directory layouts. An option to improve this behavior would be for the append file to grab the first component layout and extend to EOF, to avoid the "instantiate many stripes", while keeping some semblance of control in the hands of the users and consistency with other files in the directory (pool, etc.).  Together with the append flag in the layout, it would be (more) clear to users why this is happening, instead of the file layout suddenly changing to some random layout defined only on the MDS.

            adilger Andreas Dilger added a comment - One issue that was raised at LUG with the append-only layouts is that they are confusing for users, since they don't follow normal per-directory layouts. An option to improve this behavior would be for the append file to grab the first component layout and extend to EOF, to avoid the "instantiate many stripes", while keeping some semblance of control in the hands of the users and consistency with other files in the directory (pool, etc.).  Together with the append flag in the layout, it would be (more) clear to users why this is happening, instead of the file layout suddenly changing to some random layout defined only on the MDS.

            "Patrick Farrell <pfarrell@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58981
            Subject: LU-17159 lod: add dah_flags to dt_allocation_hint
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 8af7264fc653354262636db2ea81d56c555e8e8d

            gerrit Gerrit Updater added a comment - "Patrick Farrell <pfarrell@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58981 Subject: LU-17159 lod: add dah_flags to dt_allocation_hint Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 8af7264fc653354262636db2ea81d56c555e8e8d

            People

              paf0186 Patrick Farrell
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated: