Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-11784

PFL layouts can exceed EA size limits

Details

    • Bug
    • Resolution: Duplicate
    • Minor
    • None
    • None
    • 3
    • 9223372036854775807

    Description

      The current code for limiting layout sizes is not fully updated for PFL.  In particular, some of the limits use LOV_MAX_STRIPE_COUNT and the size of a stripe to fix the maximum layout size.  This breaks down with multiple components.

      The overstriping feature in LU-9846 makes it easy to reach high stripe counts, and so makes it more urgent to address this issue.

      This ticket is to work out what the actual current limit on EA size is (there are several different ones documented in the code, and Lustre crashes below all of them) and update the layout code to not allow creation of layouts that exceed it.

      Attachments

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. LU-9846 Overstriping - more than stripe per OST per component

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            [LU-11784] PFL layouts can exceed EA size limits
            adilger Andreas Dilger made changes -
            Resolution New: Duplicate [ 3 ]
            Status Original: Open [ 1 ] New: Resolved [ 5 ]
            adilger Andreas Dilger added a comment -

            I think this was resolved with LU-9846.

            adilger Andreas Dilger added a comment - I think this was resolved with LU-9846 .
            adilger Andreas Dilger added a comment - - edited

            Patrick, is this ticket still needed, or were the "large xattr" issues mostly addressed? Are we sticking with the "2000 stripe" maximum (about 48KB for plain RAID-0 layouts), or do we want to bite the bullet and allow a layout that is closer to the 64KB xattr size limit (possibly reserving some space so that the whole MDS reply can still fit into a 64KB buffer)?

            That would give us closer to 2500-2600 stripes, maybe 2560 stripes = 61504 bytes reasonable maximum to shoot for, leaving 4032 bytes for other parts of the RPC.

            Not that this is an urgent issue to pay attention to, just trying to close the ticket if it isn't needed.

            adilger Andreas Dilger added a comment - - edited Patrick, is this ticket still needed, or were the "large xattr" issues mostly addressed? Are we sticking with the "2000 stripe" maximum (about 48KB for plain RAID-0 layouts), or do we want to bite the bullet and allow a layout that is closer to the 64KB xattr size limit (possibly reserving some space so that the whole MDS reply can still fit into a 64KB buffer)? That would give us closer to 2500-2600 stripes, maybe 2560 stripes = 61504 bytes reasonable maximum to shoot for, leaving 4032 bytes for other parts of the RPC. Not that this is an urgent issue to pay attention to, just trying to close the ticket if it isn't needed.
            adilger Andreas Dilger made changes -
            Link New: This issue is related to DDN-645 [ DDN-645 ]
            jgmitter Joseph Gmitter (Inactive) made changes -
            Labels Original: pfl New: PFL2
            jgmitter Joseph Gmitter (Inactive) made changes -
            Labels New: pfl
            paf Patrick Farrell (Inactive) made changes -
            Link New: This issue is related to LU-9846 [ LU-9846 ]
            paf Patrick Farrell (Inactive) created issue -

            People

              paf Patrick Farrell (Inactive)
              paf Patrick Farrell (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: