[LU-18109] Nodemap UID/GID/PROJID idmap offsetting

• Type: New Feature
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.17.0
• Severity: 3
• Assignee: Max Dilger
• Reporter: Max Dilger

Description

For multi-tenancy configuration on a Lustre filesystem, it would be preferable to allow multiple Tenants using a different user authentication service that is owned by the Tenant, eg: an LDAP per tenant.

In this scenario, Tenants may have many tens or hundreds of users and groups defined, and these will be highly dynamic, with users and groups being added and removed. These UIDs, GIDs, and PROJIDs may overlap across tenants, and we do not have the possibility of dictating what UID/GID/PROJID ranges each tenant may allocate from.

To support such an environment, the storage administrator could create an individual mapping for each and every ID used by a Tenant, however this would be extremely onerous on the Storage administrator to maintain and keep up to date, and it the Storage Administrator may not have any access or knowledge about the Tenant IDs, working more as a Cloud Service Provider, separate from the team managing the Tenant). As well, having hundreds of thousands or millions of different IDs to map at the server level will put strain on the server resources (memory, CPU).

Instead, could we add a new configuration option to a nodemap to specify an ID mapping offset start and end, so that automatically, every Tenant local ID would be mapped to a new value "UID+OFFSET" on the storage, without any explicit mapping being set.

Then a storage administrator could choose to allocate 1M canonical UIDs to a Tenant, by specifying this property:

Tenant1: canonical UID = UID+100,000
Tenant2: canonical UID = UID+200,000
Tenant3: ...

This way each tenant, will have distinct canonical UID/GID/PROJID range on the storage, without any effort to maintain the mapping rules as new users are created/deleted within the Tenants.

As discussed with adilger and sebastien all nodemaps will be able to have their UID/GID/PROJID offset set individually, but by default will all be set the same. The offset values will also only be applied to the individual FSID values when they are being loaded, and the original mapped values are what are saved. For ease of use, after the offset is declared users will not need to add it to every mapping (i.e. the admin will specify CLID:FSID as if offset didn't exist), and the offset will be automatically added in the back end when the nodemap handled.

Issue Links

• is related to: LU-18222 Implement quota aggregation limits by ID range
• is related to: LU-18247 Nodemap does not initialize unused fields on disk
• is related to: LU-18744 sanity-sec test_27ab: lfs setquota failed
• is related to: LU-17166 add NRS TBF rule for projid
• is related to: LU-18775 HSM archive ID mapping for nodemaps
• is related to: LU-17922 nodemap should allow declaring ID ranges

Comments

[Peter Jones]

Merged for 2.17

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57856/
Subject: LU-18109 nodemap: fix idmap offset for root
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: b162043239646ca7a26da2146b56578b5ed0d3af

[Sebastien Buisson]

I think as soon as patch "LU-18109 nodemap: fix idmap offset for root" https://review.whamcloud.com/57856 gets merged, we can mark this ticket resolved.

[Andreas Dilger]

Is there anything left in this ticket, or should it be marked resolved?

[Gerrit Updater]

"Sebastien Buisson <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57856
Subject: LU-18109 nodemap: fix idmap offset for root
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: fdce00c814c494f995e9db7b74e301601a516243

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/57555/
Subject: LU-18109 tests: skip sanity-sec test_27ab for old MDS
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 225440c921cb45dc8652aca712bed501fa61b5f0

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/55943/
Subject: LU-18109 utils: adding nodemap offset capability
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: e3051ad0f1c8c3399c33eaf7cb0cc906177f7891

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/57555
Subject: LU-18109 tests: skip sanity-sec test_27ab for old MDS
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: b2ca295773b01a7e9d59145142a50391a211317d

[Matt Rásó-Barnett]

My own preference is for the first option. My reasoning is that the primary purpose for the offset range is to ensure that each nodemap has a dedicated ID space, so in my opinion, any other regular idmaps should be applied within that ID space.

I might be missing some use-cases where IDs need to be mapped outside of this range, but I can't think of any currently.

[Andreas Dilger]

sthiell, mrasobarnett, eaujames, gmason-nvda, rdruon,adegremont_nvda
Sebastien, Max and I are soliciting input on how nodemap ID offsets should be applied to IDs that are also being mapped with a regular idmap or squash. In the case of a nodemap having an offset range (eg. mapping ID 0-99999 for some sub-tenant to 300000-399999) should it be:

• the idmap and squash values are specified in the pre-offset (client) ID space, and then always mapped to the offset range?
• the idmap and squash values are specified so that they map to the post-offset (filesystem) range and never mapped?
• the idmap and squash values could be specified in either range, and if the target values are below the limit (< 99999) then they will be mapped, otherwise left as-is?

The benefit of the first option is that the IDs are specified consistently (eg. 65534 for "nobody") and in the ID space that the end-user would be using (eg. "map UID 1000 to 3456") and would always be offset to the tenant range (regardless of what it is for that tenant).

The benefit of the second option is that the IDs would be exactly what the storage admin specifies and not offset again, but this would need to be done explicitly by the storage admin for each different tenant offset range (ie. add the offset and set "squash = 365534" for that tenant and "map UID = 303456").

The third option would try to handle this automatically, which might be easier, but has some risk that it would do something unexpected if the storage admin was trying to map an ID and made a mistake... I list it for completeness, but I don't think it is anyone's first choice.