Details

    • Improvement
    • Resolution: Unresolved
    • Minor
    • None
    • Lustre 2.14.0, Lustre 2.16.0
    • None
    • 3
    • 9223372036854775807

    Description

      A number of sites using containers with Lustre have requested the ability to dynamically configure nodemaps on a regular basis to handle environments where they are being created and removed on a e.g. a per-job basis to contain only the nodes in the job for a specific runtime environment.

      Storing all of the nodemaps in the config llog with frequent updates would quickly consume all of the space in the config log, and be slow to process during mounting.

      Instead, there should be some option to allow using "lctl set_param nodemap.NODEMAP.PARAM" or an option to "lctl nodemap" to directly create and manage nodemaps without using the config llog to distribute the parameters to each server. This would require some external orchestration to set the nodemap parameters consistently across servers (e.g. "clush -a lctl set_param ...").

      It might be desirable to have these dynamic nodemaps inherit settings from an existing persistent nodemap that would otherwise catch their NID range. This would have a number of benefits:

      • safety in case of server restart so that missing dynamic nodemaps do not result in clients in the default nodemap
      • avoid the need to set every parameter for the new nodemap, only change the subdirectory and NIDs and possibly a few other parameters
      • (optionally?) make the dynamic nodemaps "safe by default" in that they can only add further restrictions rather than weaken them (e.g. longer subdirectory path, only disable access permissions, only remove UID mappings from the parent). Not totally sure about this, but seems interesting.

      For identifying at mount time which nodemap is used, this could be hierarchical. Find the top-level NID range as today and map it to a nodemap, and consider this the "parent" nodemap. Then check if there is a second-level NID range below the parent and re-scan for matching NIDs and add extra restrictions from the child nodemap. Possibly repeat (max 10 levels for safety?).

      That avoids making the initial NID searching complex (broad NID ranges can be used for the whole cluster) and then become more finegrained only if needed.

      Attachments

        Issue Links

          Activity

            [LU-17431] dynamically configurable nodemap
            gerrit Gerrit Updater added a comment -

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/59918/
            Subject: LU-17431 nodemap: rbac roles in child_raise_privileges
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 96441c754911e8a4ae6bd098d30cf2c3b07808d9

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/59918/ Subject: LU-17431 nodemap: rbac roles in child_raise_privileges Project: fs/lustre-release Branch: master Current Patch Set: Commit: 96441c754911e8a4ae6bd098d30cf2c3b07808d9

            "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/60044
            Subject: LU-17431 nodemap: do not revoke global lock for dyn nodemap
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 9209dcf2988cb1dbd1b37496c055df2f7364835c

            gerrit Gerrit Updater added a comment - "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/60044 Subject: LU-17431 nodemap: do not revoke global lock for dyn nodemap Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 9209dcf2988cb1dbd1b37496c055df2f7364835c

            "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/59999
            Subject: LU-17431 nodemap: Fileset modification on dynamic nodemaps
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: be3a54f010bd637f4fae007be88538941012ba49

            gerrit Gerrit Updater added a comment - "Marc Vef <mvef@whamcloud.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/59999 Subject: LU-17431 nodemap: Fileset modification on dynamic nodemaps Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: be3a54f010bd637f4fae007be88538941012ba49

            "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/59918
            Subject: LU-17431 nodemap: rbac roles in child_raise_privileges
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 842d5ee143e81fedb24a4ed3e07690d1d5381464

            gerrit Gerrit Updater added a comment - "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/59918 Subject: LU-17431 nodemap: rbac roles in child_raise_privileges Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 842d5ee143e81fedb24a4ed3e07690d1d5381464

            "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/59892
            Subject: LU-17431 nodemap: reclassify members
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 99f4a1019b8587008a4641daa1581ab2134cdc33

            gerrit Gerrit Updater added a comment - "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/59892 Subject: LU-17431 nodemap: reclassify members Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 99f4a1019b8587008a4641daa1581ab2134cdc33

            "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/59890
            Subject: LU-17431 nodemap: find innermost nid range
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 98c66ec24e567edad402841bb8610ea13769da8d

            gerrit Gerrit Updater added a comment - "Sebastien Buisson <sbuisson@ddn.com>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/59890 Subject: LU-17431 nodemap: find innermost nid range Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 98c66ec24e567edad402841bb8610ea13769da8d

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55539/
            Subject: LU-17431 nodemap: introduce child_raise_privileges property
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 904a7f1e8396eaa4bb969ccf0214ac0591f8887e

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55539/ Subject: LU-17431 nodemap: introduce child_raise_privileges property Project: fs/lustre-release Branch: master Current Patch Set: Commit: 904a7f1e8396eaa4bb969ccf0214ac0591f8887e

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55002/
            Subject: LU-17431 tests: exercise dynamic nodemap on MDS
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 7dd4b9d9b650a505fec2fffe559f836d89fc7c55

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55002/ Subject: LU-17431 tests: exercise dynamic nodemap on MDS Project: fs/lustre-release Branch: master Current Patch Set: Commit: 7dd4b9d9b650a505fec2fffe559f836d89fc7c55

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54739/
            Subject: LU-17431 nodemap: make dynamic nodemaps hierarchical
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: e9278b8da16338db30ac34abff1f6bc3489352b0

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54739/ Subject: LU-17431 nodemap: make dynamic nodemaps hierarchical Project: fs/lustre-release Branch: master Current Patch Set: Commit: e9278b8da16338db30ac34abff1f6bc3489352b0

            "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58088/
            Subject: LU-17431 nodemap: modify all properties of a dynamic nm
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 976a14cd8c5c805b10294a921eec7a943d80bf47

            gerrit Gerrit Updater added a comment - "Oleg Drokin <green@whamcloud.com>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58088/ Subject: LU-17431 nodemap: modify all properties of a dynamic nm Project: fs/lustre-release Branch: master Current Patch Set: Commit: 976a14cd8c5c805b10294a921eec7a943d80bf47

            People

              sebastien Sebastien Buisson
              adilger Andreas Dilger
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated: