Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-11057

Client mount option "-o network=net" does not work with LNet dynamic peer discovery

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • Lustre 2.12.0
    • Lustre 2.11.0
    • 3
    • 9223372036854775807

    Description

      Hi,

      It appears the client mount option "-o network=net" does not work with LNet dynamic peer discovery.
      For the record, the purpose of the "network" client mount option is to limit connections from the client to be on the network NID specified by 'net' ('net' designates a single network NID, like 'o2ib2' or 'tcp1').
      This feature was landed in 2.10:
      https://review.whamcloud.com/19792
      This option can be useful in case of several Lustre client mount points on the same node, with each mount point using a different network. It is also interesting when running Lustre clients from containers, by restricting each container to a specific network.
      In fact, this client mount option is decisive to implement multi-tenancy with Lustre.

      When LNet dynamic peer discovery is enabled, the "-o network=net" client mount option has no effect.
      When LNet dynamic peer discovery is disabled cluster-wide (either by issuing "lnetctl set discovery 0" on all nodes, or adding line "options lnet lnet_peer_discovery_disabled=1" to modprobe.conf on all nodes), then "-o network=net" client mount option is taken into account.

      This is a problem as LNet dynamic peer discovery is enabled by default starting with Lustre 2.11.
      The "-o network=net" client mount option should not be ignored, this is considered as a regression. Even when LNet dynamic peer discovery is enabled, it should be possible to restrict a client import to only use a given NID.

      I have looked into the code, but with the recent features landed, LNet code has become more complicated, and I am not able to figure out how to re-implement client-side network restriction.

      Thanks,
      Sebastien.

      Attachments

        Issue Links

          Activity

            [LU-11057] Client mount option "-o network=net" does not work with LNet dynamic peer discovery
            pjones Peter Jones added a comment -

            Landed for 2.12

            pjones Peter Jones added a comment - Landed for 2.12

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33189/
            Subject: LU-11057 tests: exercise 'network' client mount option
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 4e429959cb9943a98daee9aedb0bfa1853d8d625

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33189/ Subject: LU-11057 tests: exercise 'network' client mount option Project: fs/lustre-release Branch: master Current Patch Set: Commit: 4e429959cb9943a98daee9aedb0bfa1853d8d625

            Sebastien Buisson (sbuisson@ddn.com) uploaded a new patch: https://review.whamcloud.com/33189
            Subject: LU-11057 tests: exercise 'network' client mount option
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 33329534df9b6996d82d2f1d29c7d2f0947f6ec2

            gerrit Gerrit Updater added a comment - Sebastien Buisson (sbuisson@ddn.com) uploaded a new patch: https://review.whamcloud.com/33189 Subject: LU-11057 tests: exercise 'network' client mount option Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 33329534df9b6996d82d2f1d29c7d2f0947f6ec2

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33139/
            Subject: Revert "LU-11057 tests: exercise 'network' client mount option"
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 3d05ffdca87fb96f7971b612e8ed7455500a8826

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/33139/ Subject: Revert " LU-11057 tests: exercise 'network' client mount option" Project: fs/lustre-release Branch: master Current Patch Set: Commit: 3d05ffdca87fb96f7971b612e8ed7455500a8826

            Oleg Drokin (green@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/33139
            Subject: Revert "LU-11057 tests: exercise 'network' client mount option"
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: 55ebb1ce7c419ee358ced64a5daeba9afa31afed

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) uploaded a new patch: https://review.whamcloud.com/33139 Subject: Revert " LU-11057 tests: exercise 'network' client mount option" Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: 55ebb1ce7c419ee358ced64a5daeba9afa31afed
            pjones Peter Jones added a comment -

            Landed for 2.12

            pjones Peter Jones added a comment - Landed for 2.12

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/32590/
            Subject: LU-11057 tests: exercise 'network' client mount option
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 76af9f62481b1ac910cd6a0f7237fe32b0657c29

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/32590/ Subject: LU-11057 tests: exercise 'network' client mount option Project: fs/lustre-release Branch: master Current Patch Set: Commit: 76af9f62481b1ac910cd6a0f7237fe32b0657c29

            Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/32562/
            Subject: LU-11057 obd: check '-o network' and peer discovery conflict
            Project: fs/lustre-release
            Branch: master
            Current Patch Set:
            Commit: 2269d27e07cb4dd9c80a770dec45fa6bd22883ab

            gerrit Gerrit Updater added a comment - Oleg Drokin (green@whamcloud.com) merged in patch https://review.whamcloud.com/32562/ Subject: LU-11057 obd: check '-o network' and peer discovery conflict Project: fs/lustre-release Branch: master Current Patch Set: Commit: 2269d27e07cb4dd9c80a770dec45fa6bd22883ab

            Carrying out some more tests, I realized that the exports list as seen under /proc/fs/lustre//$fsname-/exports can be misleading when LNet dynamic peer discovery is on. Indeed, the 'name' used for the export does not assume anything on the actual connection used.

            So it turns out the '-o network' mount option is taken into account as soon as LNet dynamic peer discovery is deactivated on the local client node (meaning there is no need to have it disabled cluster-wide, it can be handled on each node individually).

            sbuisson Sebastien Buisson (Inactive) added a comment - Carrying out some more tests, I realized that the exports list as seen under /proc/fs/lustre/ /$fsname- /exports can be misleading when LNet dynamic peer discovery is on. Indeed, the 'name' used for the export does not assume anything on the actual connection used. So it turns out the '-o network' mount option is taken into account as soon as LNet dynamic peer discovery is deactivated on the local client node (meaning there is no need to have it disabled cluster-wide, it can be handled on each node individually).

            Sebastien Buisson (sbuisson@ddn.com) uploaded a new patch: https://review.whamcloud.com/32590
            Subject: LU-11057 tests: exercise 'network' client mount option
            Project: fs/lustre-release
            Branch: master
            Current Patch Set: 1
            Commit: edacb8b5b59f9bfa04681f7fe2dee05fe96fc269

            gerrit Gerrit Updater added a comment - Sebastien Buisson (sbuisson@ddn.com) uploaded a new patch: https://review.whamcloud.com/32590 Subject: LU-11057 tests: exercise 'network' client mount option Project: fs/lustre-release Branch: master Current Patch Set: 1 Commit: edacb8b5b59f9bfa04681f7fe2dee05fe96fc269

            People

              sebastien Sebastien Buisson
              sbuisson Sebastien Buisson (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: