Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-18357

Support multiple filesets per nodemap

Details

    • New Feature
    • Resolution: Unresolved
    • Minor
    • None
    • None
    • 9223372036854775807

    Description

      Multiple filesets per nodemap would be beneficial. The filesets consist of one primary path and several alternate path for a given tenant's nodemap. The primary fileset (mostly) corresponds to the current implementation's single available fileset. The proposed API, mounting rules, persistence mechanism, and update/downgrade behavior are described below:

      API

       (new) lctl nodemap_fileset_add --name <nodemap_name> [--alt] [--ro] --fileset <path>
          - Works on both types of filesets based on a flag --alt to add an alternate fileset.
          - Any fileset can be set read-only with --ro.
          - Any type of fileset can only be added once and does not overwrite - returns EEXIST if already set. If the passed path is equal to a currently stored fileset (based on whether primary/alternate filesets are addressed), no error code is set and the request is quietly ignored.
          - Only one primary fileset is allowed. Note, a primary fileset is not required to set alternate filesets and vice versa.
          - Alternate filesets are continuously added when the --alt flag is set.
      2. (new) lctl nodemap_fileset_del --name <nodemap_name> --fileset <single_fileset_path>
          - Works on both types of filesets by their path and deletes a fileset.
      3. (new) lctl nodemap_fileset_modify --name <nodemap_name> --fileset <fileset_path> [--rename <new_fileset_path>] [--ro] [--rw]
          - Works on both types of filesets by their path.
          - Used to change a fileset path (similar to rename) or flag in-place using the --rename (or similar) option.
          - Duplicate filesets are not allowed across all fileset types.
          - Flags, i.e., --ro, --rw, can be set.
      4. (new) lctl nodemap_filesets_clear --name <nodemap_name>
          - Works on all filesets and deletes all primary and alternate filesets for a nodemap.
      5. (deprecate, leave as is) nodemap_set_fileset --name <nodemap_name> --fileset <single_fileset_path>
          - This command does not seem to be useful for the future both semantically and functionally. The main concern is that we would need some format to address multiple filesets, their types, and flags all in one string if the command should be flexible. It is also a duplicate of nodemap_add_fileset. Therefore, replacing it with nodemap_fileset_add seems preferable.
          - With multiple filesets, this command would only overwrite the primary fileset without affecting alternate filesets, which this command cannot address.
      6. (extended) lctl get_param nodemap.<nodemap_name>.fileset
          - Prints all filesets and their type in a similar format as other nodemap properties that can be fetched with lctl get_param.
      7. (extended) lctl nodemap_info [--name <nodemap_name>] [--list-properties] [-–property <value>]
          - Extends the existing lctl nodemap_info command to support all nodemap properties and values including filesets. Overall, this makes the command more useful in practice as it can currently only list the property names and not their values.
          - Note, the current command uses positional parameters [<nm>|all|list]. This should remain backward compatible for some time but deprecated.

      9. (deprecate, leave as is) lctl set_param nodemap.<nm>.fileset=<path>
          - This command should be deprecated since we should use the nodemap IAM for persistence.
          - Similar to lctl nodemap_set_fileset it would only affect the primary fileset when used.

      Mounting rules

      1. If the nodemap is inactive or no filesets are set, there are no fileset restrictions on the mounting subdir.
      2. If no mounting subdir is set, the primary fileset is used and the mounting subdir is replaced by it.
      3. If any fileset (primary and alternate) fileset matches the mounting subdir exactly or as a prefix, the client is permitted to mount the subdir.
      4. If no fileset was found in 3., the mounting subdir is appended to the primary fileset.
      5. In all cases (for rules 2-4), the subdir must exist to mount it.

      If no primary fileset is set, rules 2 and 4 are omitted, forcing an mount subdir that is represented by a alternate fileset.

      Persistence mechanism and update/downgrade behavior

      Similar to other nodemap properties, filesets should use the nodemap IAM records for persistence. The current mechanism to use the params llog with lctl set_param -P nodemap.<nm_name>.fileset=<fileset> is deprecated and would only affect the primary fileset. Note, it must remain in operation to allow backward-compatibility. Therefore, a previously set fileset is converted into a primary fileset on an update.

      Due to the use of the nodemap IAM for persistence, multiple filesets are not downgrade-compatible. The fileset would fall-back to any available fileset definition in the params llog.

      Attachments

        Issue Links

          Activity

            [LU-18357] Support multiple filesets per nodemap

            "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58436
            Subject: LU-18357 test: fix sanity-sec 27a interop with long fileset
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: d35e04f314d1a482ee73866047d9623977f250aa

            gerrit Gerrit Updater added a comment - "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58436 Subject: LU-18357 test: fix sanity-sec 27a interop with long fileset Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: d35e04f314d1a482ee73866047d9623977f250aa

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57600/
            Subject: LU-18357 ptlrpc: Add alternate fileset data structure handler
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: f6bb1b38f0814c2f8cab103fe228820aadbf2ee9

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57600/ Subject: LU-18357 ptlrpc: Add alternate fileset data structure handler Project: fs/lustre-release Branch: master Current Patch Set: Commit: f6bb1b38f0814c2f8cab103fe228820aadbf2ee9

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57527/
            Subject: LU-18357 ptlrpc: Add fileset ID handling on the nodemap IAM
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 2b8dca3fa31ef47c6131e4f9a7f99ee6ceabb62a

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57527/ Subject: LU-18357 ptlrpc: Add fileset ID handling on the nodemap IAM Project: fs/lustre-release Branch: master Current Patch Set: Commit: 2b8dca3fa31ef47c6131e4f9a7f99ee6ceabb62a

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57523/
            Subject: LU-18357 ptlrpc: Use dynamic memory allocation for filesets
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 3277457b898c3eed62e3be3fb00970e49baaf445

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57523/ Subject: LU-18357 ptlrpc: Use dynamic memory allocation for filesets Project: fs/lustre-release Branch: master Current Patch Set: Commit: 3277457b898c3eed62e3be3fb00970e49baaf445

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56757/
            Subject: LU-18357 ptlrpc: Use nodemap IAM for persisting filesets
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: da3536da22aba6b61da2c82b71e4eec423425446

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/56757/ Subject: LU-18357 ptlrpc: Use nodemap IAM for persisting filesets Project: fs/lustre-release Branch: master Current Patch Set: Commit: da3536da22aba6b61da2c82b71e4eec423425446

            "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58192
            Subject: LU-18357 ptlrpc: Add in-place modification of filesets
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: f24e616a05508921417c1db74ee0de714920a0ac

            gerrit Gerrit Updater added a comment - "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58192 Subject: LU-18357 ptlrpc: Add in-place modification of filesets Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: f24e616a05508921417c1db74ee0de714920a0ac

            "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57900
            Subject: LU-18357 ptlrpc: Support clearing all filesets from a nodemap
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 4b073314d735db2d955cbf15c07877f7ee8de676

            gerrit Gerrit Updater added a comment - "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57900 Subject: LU-18357 ptlrpc: Support clearing all filesets from a nodemap Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 4b073314d735db2d955cbf15c07877f7ee8de676

            "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57829
            Subject: LU-18357 ptlrpc: Add read-only flag for nodemap filesets
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 41eb68e7f0d0069067897040663dc02f007500f5

            gerrit Gerrit Updater added a comment - "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57829 Subject: LU-18357 ptlrpc: Add read-only flag for nodemap filesets Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 41eb68e7f0d0069067897040663dc02f007500f5

            "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57609
            Subject: LU-18357 ptlrpc: Multiple fileset support on client mount
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: ba987d5a7d180e6fa3d1a2ee0dd832a9ec318a84

            gerrit Gerrit Updater added a comment - "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57609 Subject: LU-18357 ptlrpc: Multiple fileset support on client mount Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: ba987d5a7d180e6fa3d1a2ee0dd832a9ec318a84

            "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57600
            Subject: LU-18357 ptlrpc: Add alternate fileset data structure handler
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 5be584177c76ba35e08dec475d304934c89c985f

            gerrit Gerrit Updater added a comment - "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57600 Subject: LU-18357 ptlrpc: Add alternate fileset data structure handler Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 5be584177c76ba35e08dec475d304934c89c985f

            "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57527
            Subject: LU-18357 ptlrpc: Add fileset ID handling on the nodemap IAM
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 9ffaee0984d386804ff36bacb9eafd06f0b207c2

            gerrit Gerrit Updater added a comment - "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57527 Subject: LU-18357 ptlrpc: Add fileset ID handling on the nodemap IAM Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 9ffaee0984d386804ff36bacb9eafd06f0b207c2

            People

              mvef Marc Vef
              mvef Marc Vef
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: