[LU-17431] dynamically configurable nodemap

• Type: Improvement
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.14.0, Lustre 2.16.0
• Severity: 3
• Assignee: Sebastien Buisson
• Reporter: Andreas Dilger

Description

A number of sites using containers with Lustre have requested the ability to dynamically configure nodemaps on a regular basis to handle environments where they are being created and removed on a e.g. a per-job basis to contain only the nodes in the job for a specific runtime environment.

Storing all of the nodemaps in the config llog with frequent updates would quickly consume all of the space in the config log, and be slow to process during mounting.

Instead, there should be some option to allow using "lctl set_param nodemap.NODEMAP.PARAM" or an option to "lctl nodemap" to directly create and manage nodemaps without using the config llog to distribute the parameters to each server. This would require some external orchestration to set the nodemap parameters consistently across servers (e.g. "clush -a lctl set_param ...").

It might be desirable to have these dynamic nodemaps inherit settings from an existing persistent nodemap that would otherwise catch their NID range. This would have a number of benefits:

• safety in case of server restart so that missing dynamic nodemaps do not result in clients in the default nodemap
• avoid the need to set every parameter for the new nodemap, only change the subdirectory and NIDs and possibly a few other parameters
• (optionally?) make the dynamic nodemaps "safe by default" in that they can only add further restrictions rather than weaken them (e.g. longer subdirectory path, only disable access permissions, only remove UID mappings from the parent). Not totally sure about this, but seems interesting.

For identifying at mount time which nodemap is used, this could be hierarchical. Find the top-level NID range as today and map it to a nodemap, and consider this the "parent" nodemap. Then check if there is a second-level NID range below the parent and re-scan for matching NIDs and add extra restrictions from the child nodemap. Possibly repeat (max 10 levels for safety?).

That avoids making the initial NID searching complex (broad NID ranges can be used for the whole cluster) and then become more finegrained only if needed.

Issue Links

• is related to: LU-14657 nodemap asynchronous update creates inconsistent entries.
• is related to: LU-17217 Allow server to control/deny client connections
• is related to: LU-8229 SSK: allow multiple keys on a single nodemap
• is related to: LU-17410 Add per-nodemap capabilities mask
• is related to: LU-8946 lctl nodemap_add/del_range should accept non-contiguous ranges

Comments

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55539/
Subject: LU-17431 nodemap: introduce child_raise_privileges property
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 904a7f1e8396eaa4bb969ccf0214ac0591f8887e

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55002/
Subject: LU-17431 tests: exercise dynamic nodemap on MDS
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 7dd4b9d9b650a505fec2fffe559f836d89fc7c55

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54739/
Subject: LU-17431 nodemap: make dynamic nodemaps hierarchical
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e9278b8da16338db30ac34abff1f6bc3489352b0

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/58088/
Subject: LU-17431 nodemap: modify all properties of a dynamic nm
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 976a14cd8c5c805b10294a921eec7a943d80bf47

[Gerrit Updater]

"Sebastien Buisson <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/58088
Subject: LU-17431 nodemap: modify all properties of a dynamic nm
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 8a5bcb51ac674da4f3335163c5cae148e5c702d9

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/54742/
Subject: LU-17431 nodemap: introduce func to init nodemap properties
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 44565cbe9d3ba36d796fd4827dfcabe33382b40e