[LU-11057] Client mount option "-o network=net" does not work with LNet dynamic peer discovery

Related — Wed, 15 Feb 2023 09:26:39 +0000

Hi,

It appears the client mount option "-o network=net" does not work with LNet dynamic peer discovery.
For the record, the purpose of the "network" client mount option is to limit connections from the client to be on the network NID specified by 'net' ('net' designates a single network NID, like 'o2ib2' or 'tcp1').
This feature was landed in 2.10:
https://review.whamcloud.com/19792
This option can be useful in case of several Lustre client mount points on the same node, with each mount point using a different network. It is also interesting when running Lustre clients from containers, by restricting each container to a specific network.
In fact, this client mount option is decisive to implement multi-tenancy with Lustre.

When LNet dynamic peer discovery is enabled, the "-o network=net" client mount option has no effect.
When LNet dynamic peer discovery is disabled cluster-wide (either by issuing "lnetctl set discovery 0" on all nodes, or adding line "options lnet lnet_peer_discovery_disabled=1" to modprobe.conf on all nodes), then "-o network=net" client mount option is taken into account.

This is a problem as LNet dynamic peer discovery is enabled by default starting with Lustre 2.11.
The "-o network=net" client mount option should not be ignored, this is considered as a regression. Even when LNet dynamic peer discovery is enabled, it should be possible to restrict a client import to only use a given NID.

I have looked into the code, but with the recent features landed, LNet code has become more complicated, and I am not able to figure out how to re-implement client-side network restriction.

Thanks,
Sebastien.

Whamcloud Community JIRA

[LU-11057] Client mount option "-o network=net" does not work with LNet dynamic peer discovery