Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • None
    • 3
    • 9223372036854775807

    Description

      Most of the time (unless the filesystem is full), RESTORE and REMOVE requests should be processed first as they have the highest priority from a user's point of view ; ARCHIVE requests should have a lower priority.

      Attachments

        Issue Links

          Activity

            [LU-8324] HSM: prioritize HSM requests

            Quentin Bouget (quentin.bouget@cea.fr) uploaded a new patch: https://review.whamcloud.com/33239
            Subject: LU-8324 hsm: prioritize one RESTORE once in a while
            Project: fs/lustre-release
            Branch: b2_10
            Current Patch Set: 1
            Commit: 93364e9f3b0c9694904d2c1e2a687af61a980c1f

            gerrit Gerrit Updater added a comment - Quentin Bouget (quentin.bouget@cea.fr) uploaded a new patch: https://review.whamcloud.com/33239 Subject: LU-8324 hsm: prioritize one RESTORE once in a while Project: fs/lustre-release Branch: b2_10 Current Patch Set: 1 Commit: 93364e9f3b0c9694904d2c1e2a687af61a980c1f

            The patch above is the shortest/simplest hack I could come up with to help bear with LU-8324 until a more definitive fix is developed (it is more of a band-aid than anything else).

            The idea is to use the times when the coordinator traverses its whole llog to "force-schedule" at least one RESTORE request. In practice, this means that you should see at least one RESTORE request scheduled every "loop_period" (the value in /proc/<fsname>/mdt/<mdt-name>/hsm/loop_period) seconds.

            bougetq Quentin Bouget (Inactive) added a comment - The patch above is the shortest/simplest hack I could come up with to help bear with LU-8324 until a more definitive fix is developed (it is more of a band-aid than anything else). The idea is to use the times when the coordinator traverses its whole llog to "force-schedule" at least one RESTORE request. In practice, this means that you should see at least one RESTORE request scheduled every "loop_period" (the value in /proc/<fsname>/mdt/<mdt-name>/hsm/loop_period ) seconds.

            Quentin Bouget (quentin.bouget@cea.fr) uploaded a new patch: https://review.whamcloud.com/31723
            Subject: LU-8324 hsm: prioritize one RESTORE once in a while
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: b30b036607a2bc4928e13e06462701bf5ba62d3d

            gerrit Gerrit Updater added a comment - Quentin Bouget (quentin.bouget@cea.fr) uploaded a new patch: https://review.whamcloud.com/31723 Subject: LU-8324 hsm: prioritize one RESTORE once in a while Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: b30b036607a2bc4928e13e06462701bf5ba62d3d

            This thread looks kind of dead, but we have a desire to see some prioritization mechanism as well.
            Some options:
            1. FIFO (today)
            2. Restore-first. All restore requests are prioritized over archive requests. (Except in-progress archives.)
            3. Archive-first. All archives are prioritized.
            4. Interleaved. Archive and Restore requests are alternated, as long as some of each are waiting.
            5. Tunable. Adjustable ratio of archive:restore processing. Maybe this covers the above 2-4 as well.
            6. Batched. Archives and Restores are grouped into separate batches, potentially resulting in fewer tape swaps.
            7. Time-boxed. A variant of batched; batch ends after a fixed time period.
            Many other options I'm sure...

            Ultimately I'm in agreement with Robert Read's comment above that the prioritization should really be done outside of Lustre, but if the patch here implements #5 that might cover enough of the use cases to make most people happy...

            nrutman Nathan Rutman added a comment - This thread looks kind of dead, but we have a desire to see some prioritization mechanism as well. Some options: 1. FIFO (today) 2. Restore-first. All restore requests are prioritized over archive requests. (Except in-progress archives.) 3. Archive-first. All archives are prioritized. 4. Interleaved. Archive and Restore requests are alternated, as long as some of each are waiting. 5. Tunable. Adjustable ratio of archive:restore processing. Maybe this covers the above 2-4 as well. 6. Batched. Archives and Restores are grouped into separate batches, potentially resulting in fewer tape swaps. 7. Time-boxed. A variant of batched; batch ends after a fixed time period. Many other options I'm sure... Ultimately I'm in agreement with Robert Read's comment above that the prioritization should really be done outside of Lustre, but if the patch here implements #5 that might cover enough of the use cases to make most people happy...

            Quentin Bouget (quentin.bouget@cea.fr) uploaded a new patch: https://review.whamcloud.com/27800
            Subject: LU-8324 hsm: ease the development of a different coordinator
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 0ee6e5d71c6f549b48be59607d5f55f28e950f47

            gerrit Gerrit Updater added a comment - Quentin Bouget (quentin.bouget@cea.fr) uploaded a new patch: https://review.whamcloud.com/27800 Subject: LU-8324 hsm: ease the development of a different coordinator Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 0ee6e5d71c6f549b48be59607d5f55f28e950f47

            Quentin Bouget (quentin.bouget@cea.fr) uploaded a new patch: https://review.whamcloud.com/27394
            Subject: LU-8324 hsm: prioritize HSM requests
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: f8d7f289866c2219d19a51693ae44cc6c3fdf867

            gerrit Gerrit Updater added a comment - Quentin Bouget (quentin.bouget@cea.fr) uploaded a new patch: https://review.whamcloud.com/27394 Subject: LU-8324 hsm: prioritize HSM requests Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: f8d7f289866c2219d19a51693ae44cc6c3fdf867
            pjones Peter Jones added a comment -

            I think that 2.10.1 is more likely option at this stage. It seems like there will be some discussions about this area at LUG next week.

            pjones Peter Jones added a comment - I think that 2.10.1 is more likely option at this stage. It seems like there will be some discussions about this area at LUG next week.
            spitzcor Cory Spitz added a comment -

            If not 2.10, it seems that 2.10.1 would be possible.

            spitzcor Cory Spitz added a comment - If not 2.10, it seems that 2.10.1 would be possible.

            Any chance for this patch to make it into 2.10? It is a very useful feature for HSM users and we believe that the patch is mature.

            hdoreau Henri Doreau (Inactive) added a comment - Any chance for this patch to make it into 2.10? It is a very useful feature for HSM users and we believe that the patch is mature.

            Hello Matt,

            I think the patch is mature enough for you to test it if you are still interested in it.

            bougetq Quentin Bouget (Inactive) added a comment - Hello Matt, I think the patch is mature enough for you to test it if you are still interested in it.

            Hello,
            I would just like to say that we would be very keen on this kind of feature at Cambridge - I've just run into this issue today where a single file restore operation is at the back of the queue behind ~10TB of archive jobs.

            I'd be interested in testing this patch against one of our test filesystems, but I just wanted to add a comment that we would really appreciate having more ability to control the coordinator queue - whether it's in it's current state or some future tool as Robert suggests.

            Kind regards,
            Matt Raso-Barnett
            University of Cambridge

            mrb Matt Rásó-Barnett (Inactive) added a comment - Hello, I would just like to say that we would be very keen on this kind of feature at Cambridge - I've just run into this issue today where a single file restore operation is at the back of the queue behind ~10TB of archive jobs. I'd be interested in testing this patch against one of our test filesystems, but I just wanted to add a comment that we would really appreciate having more ability to control the coordinator queue - whether it's in it's current state or some future tool as Robert suggests. Kind regards, Matt Raso-Barnett University of Cambridge

            People

              bougetq Quentin Bouget (Inactive)
              cealustre CEA
              Votes:
              2 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated: