Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-9846

Overstriping - more than stripe per OST per component

Details

    • New Feature
    • Resolution: Fixed
    • Minor
    • Lustre 2.13.0
    • None
    • 3
    • 9223372036854775807

    Description

      'Overstriping' is my term for allowing more than one stripe of a particular file component to be placed on a particular OST.

      Justification:
      Lock ahead is designed to address the case were we are limited to a single shared file, but each OST is significantly faster than one client. In a shared file situation, LDLM locking behavior limits us to writing with one client to each OST, so we are unable to fully drive each OST for the shared file. This is an interesting enough case for Cray to drive all the work in lock ahead use a library to achieve it.

      If we can put multiple stripes of the file on a single OST, we can essentially achieve the same thing, with far less effort. For a variety of reasons, this doesn't remove the need for lockahead (Primarily because we cannot necessarily redefine file striping when we want to write to it), but it is much simpler, and highly desirable for that reason. In addition to the MPIIO aggregation case where we have well controlled I/O and are trying to maximize OST utilization, adding more stripes to a shared file also helps in cases where I/O is poorly controlled, so there are effectively more locks for the badly behaved writers to contend for.

      So, in short, I think it would be very, very desirable if, in a controlled manner, we could ask for more than one stripe to be on a given OST. A simple example is something like "8 stripes but only on these 2 OSTs", giving 4 stripes per OST (and allowing 4 client writers per OST with no fancy locking work).

      A note about PFL:
      While pfl lets you create separate components using the same OST, that's not a viable solution for the case I'm looking for, which is just > 1 stripe on a given OST. The idea being to write to all stripes in parallel, but wanting > 1 writer to be pointed at each OST. PFL would require a neverending series of components to do that.

      Patch to implement this follows.

      Attachments

        Issue Links

          Activity

            [LU-9846] Overstriping - more than stripe per OST per component
            pjones Peter Jones added a comment -

            Landed for 2.13

            pjones Peter Jones added a comment - Landed for 2.13

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/28425/
            Subject: LU-9846 lod: Add overstriping support
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 591a9b4cebc510ff51f0fdb944e5a81f08fdaf62

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/28425/ Subject: LU-9846 lod: Add overstriping support Project: fs/lustre-release Branch: master Current Patch Set: Commit: 591a9b4cebc510ff51f0fdb944e5a81f08fdaf62

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/34743/
            Subject: LU-9846 obd: Add overstriping CONNECT flag
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 5d085745af43bd15e6b7ea728491600411833b2a

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/34743/ Subject: LU-9846 obd: Add overstriping CONNECT flag Project: fs/lustre-release Branch: master Current Patch Set: Commit: 5d085745af43bd15e6b7ea728491600411833b2a

            Patrick Farrell (pfarrell@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34743
            Subject: LU-9846 obd: Add overstriping CONNECT flag
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 1b17db2eee420e32944e8668c85c92d766fd6d82

            gerrit Gerrit Updater added a comment - Patrick Farrell (pfarrell@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34743 Subject: LU-9846 obd: Add overstriping CONNECT flag Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 1b17db2eee420e32944e8668c85c92d766fd6d82

            Patrick Farrell (pfarrell@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34189
            Subject: LU-9846 tests: Overstriping test
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 637dce43039fc94c5b1695a395f02f61f28162c9

            gerrit Gerrit Updater added a comment - Patrick Farrell (pfarrell@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/34189 Subject: LU-9846 tests: Overstriping test Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 637dce43039fc94c5b1695a395f02f61f28162c9

            Decided to split out raising the stripe count limit from the feature patch, as they're different and may cause different issues.

            Both still depend on landing https://jira.whamcloud.com/browse/LU-11690 first.

            paf Patrick Farrell (Inactive) added a comment - Decided to split out raising the stripe count limit from the feature patch, as they're different and may cause different issues. Both still depend on landing https://jira.whamcloud.com/browse/LU-11690  first.

            Patrick Farrell (paf@cray.com) uploaded a new patch: https://review.whamcloud.com/33871
            Subject: LU-9846 lod: Raise stripe count limit to 10K
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: ccf5aa7d2dae44ad4657aec8181a379f574e7de3

            gerrit Gerrit Updater added a comment - Patrick Farrell (paf@cray.com) uploaded a new patch: https://review.whamcloud.com/33871 Subject: LU-9846 lod: Raise stripe count limit to 10K Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: ccf5aa7d2dae44ad4657aec8181a379f574e7de3

            Patrick Farrell (paf@cray.com) uploaded a new patch: https://review.whamcloud.com/28425
            Subject: LU-9846 lod: Add overstriping support
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 5000bef6b28c3ed8de6058a935f6d22153a71510

            gerrit Gerrit Updater added a comment - Patrick Farrell (paf@cray.com) uploaded a new patch: https://review.whamcloud.com/28425 Subject: LU-9846 lod: Add overstriping support Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 5000bef6b28c3ed8de6058a935f6d22153a71510

            People

              pfarrell Patrick Farrell (Inactive)
              paf Patrick Farrell (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: