[LU-11057] Client mount option "-o network=net" does not work with LNet dynamic peer discovery

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.11.0
• Fix Version/s: Lustre 2.12.0
• Type: Bug
• Priority: Critical
• Reporter: Sebastien Buisson (Inactive)
• Assignee: Sebastien Buisson
• Resolution: Fixed
• Labels: lnet
• Severity: 3

Description

Hi,

It appears the client mount option "-o network=net" does not work with LNet dynamic peer discovery.
For the record, the purpose of the "network" client mount option is to limit connections from the client to be on the network NID specified by 'net' ('net' designates a single network NID, like 'o2ib2' or 'tcp1').
This feature was landed in 2.10:
https://review.whamcloud.com/19792
This option can be useful in case of several Lustre client mount points on the same node, with each mount point using a different network. It is also interesting when running Lustre clients from containers, by restricting each container to a specific network.
In fact, this client mount option is decisive to implement multi-tenancy with Lustre.

When LNet dynamic peer discovery is enabled, the "-o network=net" client mount option has no effect.
When LNet dynamic peer discovery is disabled cluster-wide (either by issuing "lnetctl set discovery 0" on all nodes, or adding line "options lnet lnet_peer_discovery_disabled=1" to modprobe.conf on all nodes), then "-o network=net" client mount option is taken into account.

This is a problem as LNet dynamic peer discovery is enabled by default starting with Lustre 2.11.
The "-o network=net" client mount option should not be ignored, this is considered as a regression. Even when LNet dynamic peer discovery is enabled, it should be possible to restrict a client import to only use a given NID.

I have looked into the code, but with the recent features landed, LNet code has become more complicated, and I am not able to figure out how to re-implement client-side network restriction.

Thanks,
Sebastien.

Issue Links

• is related to: LU-16557 don't skip add_conn with -o network m...
• is related to: LU-11363 sanity-sec test 31 fails with 'unabl...
• is related to: LU-14108 Mounting targets created with mkfs "n...

Comments

[Gerrit Updater]

Sebastien Buisson ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32562
Subject: LU-11057 obd: check '-o network' and peer discovery conflict
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a4a808d33a5ca4f53beee69832c526b057707fbc

[Gerrit Updater]

Sebastien Buisson ([e-mail]) uploaded a new patch: https://review.whamcloud.com/32590
Subject: LU-11057 tests: exercise 'network' client mount option
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: edacb8b5b59f9bfa04681f7fe2dee05fe96fc269

[Sebastien Buisson (Inactive)]

Carrying out some more tests, I realized that the exports list as seen under /proc/fs/lustre//$fsname-/exports can be misleading when LNet dynamic peer discovery is on. Indeed, the 'name' used for the export does not assume anything on the actual connection used.

So it turns out the '-o network' mount option is taken into account as soon as LNet dynamic peer discovery is deactivated on the local client node (meaning there is no need to have it disabled cluster-wide, it can be handled on each node individually).

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/32562/
Subject: LU-11057 obd: check '-o network' and peer discovery conflict
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 2269d27e07cb4dd9c80a770dec45fa6bd22883ab

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/32590/
Subject: LU-11057 tests: exercise 'network' client mount option
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 76af9f62481b1ac910cd6a0f7237fe32b0657c29

[Peter Jones]

Landed for 2.12

[Gerrit Updater]

Oleg Drokin ([e-mail]) uploaded a new patch: https://review.whamcloud.com/33139
Subject: Revert "LU-11057 tests: exercise 'network' client mount option"
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 55ebb1ce7c419ee358ced64a5daeba9afa31afed

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/33139/
Subject: Revert "LU-11057 tests: exercise 'network' client mount option"
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 3d05ffdca87fb96f7971b612e8ed7455500a8826

[Gerrit Updater]

Sebastien Buisson ([e-mail]) uploaded a new patch: https://review.whamcloud.com/33189
Subject: LU-11057 tests: exercise 'network' client mount option
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 33329534df9b6996d82d2f1d29c7d2f0947f6ec2

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/33189/
Subject: LU-11057 tests: exercise 'network' client mount option
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 4e429959cb9943a98daee9aedb0bfa1853d8d625

[Peter Jones]

Landed for 2.12