Problem

A Lustre client can have more than one NID. LNet selects one of these NIDs as the primary NID. The primary NID can change: the client can add or delete NIDs at run time, and LNet Discovery can correct a primary NID that a peer cached before discovery completed.

The nodemap feature classifies a client by NID. The manual and the code do not tell the administrator what must occur when a client has more than one NID, and the NID ranges of one nodemap do not contain all of these NIDs.

What the code does now

The server classifies an export by NID at two moments only:

1. At connect time, from `req->rq_peer.nid`:

• `lustre/ofd/ofd_obd.c:288` and `:359`
• `lustre/mdt/mdt_handler.c:7523` and `:7576`
• `lustre/mgs/mgs_handler.c:1845`

2. At a nodemap configuration change, from `exp->exp_connection->c_peer.nid` in `nm_member_reclassify_nodemap()` (`lustre/ptlrpc/nodemap_member.c:423`).
The callers are the range and nodemap add/delete paths and the activation path in `lustre/ptlrpc/nodemap_handler.c`.

Between these two moments, every consumer uses the cached `exp->exp_target_data.ted_nodemap` through `nodemap_get_from_exp()`
(`lustre/ptlrpc/nodemap_handler.c:1085`). No code compares the full set of NIDs of a client against the NID ranges of a nodemap. The word "primary" in the nodemap code refers to the primary fileset only.

Two mechanisms already give an identity that is stronger than the NID:

• GSS: if the reverse import has a security context with `ps_nm_name`,
  and the nodemap has `nmf_gss_identify`, the authenticated nodemap wins against the NID classification (`lustre/ptlrpc/nodemap_member.c:439-464`).
• `nodemap_member_switch()` (`lustre/ptlrpc/nodemap_handler.c:897`) lets the administrator move an export manually.

What the manual says

`LustreNodemap` states a per-operation, NID-based model:

> "Lustre connectivity is governed by network identifiers, or NIDs...
> When an operation is made from a NID, Lustre decides if that NID is > part of a nodemap... If no policy group exists for that NID, access is > squashed to user nobody by default."

It also documents that a live client moves between nodemaps when the configuration changes: "It is possible to delete a NID range from a nodemap, any clients in this NID range will be moved to the default nodemap."

The manual does not mention multi-rail or multi-NID clients. The LNet Multi-Rail chapter does not mention nodemaps. The manual does not tell the administrator to put all NIDs of a client in the same nodemap.

Example

1. The administrator puts NID A in nodemap X. NID B is not in any range of X.
2. The client connects from NID A. The server classifies the export into X.
3. The client adds NID B and deletes NID A. The primary NID becomes B.
4. The administrator makes an unrelated range change. Reclassification runs and uses the NID that the export holds.

The result depends on the cached NID, not on a documented rule. If the cached NID is A, the export stays in X although the client can no longer use A. If the cached NID is B, the export moves to the nodemap that contains B, or to the default nodemap.

Both results are defensible, but the project has not selected one.

Questions to answer

1. Which rule applies?

• Rule 1 (follow the current NID): classify from the NID that the client uses now. This rule agrees with the manual and with the documented effect of a range delete.
• Rule 2 (persist for the lifetime of the connection): keep the nodemap that the export received at connect time until the export disconnects or an administrator moves it.

2. If Rule 1 applies, must the classification also change when the primary NID changes, and not only at a configuration change?

3. Must the administrator put all NIDs of a client in the same nodemap?
If yes, must `nodemap_add_range` warn or refuse when the range splits the known NIDs of a peer?

4. What is the correct result when the NIDs of one client are in different nodemaps? Options: the nodemap of the primary NID, the most restrictive nodemap, or the default nodemap.

Possible work items

These items are for discussion. Do not do them before the rule is selected.

• Write the selected rule in the Nodemap chapter of the manual, with a statement about multi-NID clients.
• Reclassify an export when the server learns a new primary NID for the client, if Rule 1 applies. This can use `nodemap_member_switch()` or a deferred call to the reclassification path.
• Refresh `ted_nodemap` at the same moment, because a refresh of `exp_connection` alone does not correct a classification that was wrong at connect time.
• Add a warning when a nodemap range covers only a part of the NIDs of a known peer.
• Add a test that changes the primary NID of a client and checks the nodemap membership.

Relation to LU-20403

LU-20403 makes the server refresh `exp->exp_connection` when it learns that the primary NID of a client changed. That change corrects the NID that the server has in its cache. It does not change `ted_nodemap`, and it does not select a classification rule.

The example above gives an incorrect result with or without LU-20403;
only the symptom is different. The refresh makes the cached NID agree with the network state, which is the input that the reclassification path uses. This ticket covers the rule itself, which is a nodemap question and is older than LU-20403.