Problem

A per-net flavor rule, such as testfs.srpc.flavor.tcp1=skn, matches on the net of a NID. Each side uses the primary NID of the other side:

• The client: sptlrpc_import_sec_adapt() gives imp_connection->c_peer.nid to sptlrpc_conf_choose_flavor(). This step runs at each connect and reconnect.
• The server: tgt_connect_check_sptlrpc() gives req->rq_peer.nid, the primary NID of the client, to sptlrpc_target_choose_flavor().

With Multi-Rail and LNet Dynamic Peer Discovery, a primary NID can be on any net of the node. The primary NID can also change. One connection can send on many nets. Thus a per-net rule does not describe the net that the RPCs use. The two sides can also select different flavors. The server then refuses the RPCs with "unauthorized rpc flavor". A client can also use a weaker flavor than the rule for the net in use.

Example:

• A server has NIDs on tcp0 and o2ib0. Its primary NID is on o2ib0.
• The rules are srpc.flavor.tcp0=krb5p and srpc.flavor.default=null.
• A client that sends on tcp0 selects null, because the primary NID of the server is on o2ib0.

LU-20384 made per-net rules match again. Before that fix, only the default rule applied. LU-20403 removes the lock of the primary NID. After LU-20403, the client side follows the primary NID that discovery reports, as the server side already did.

This analysis is source-only. No test shows the failure yet.

Future work

Select the flavor from the net that the RPCs use, on both sides. Questions to answer:

• Can the client select a flavor per LNet message, from the net of the local NI and the peer NI? A GSS context is per import, so a per-net flavor can need a context for each net.
• Can the server check the flavor against the net of each received message, not against the primary NID of the client?
• Should a connection with a per-net rule send only on that net, as the 'network' mount option does?

Until this work lands, the manual must describe the current behavior. A separate LUDOC ticket covers that note.