Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20403

Refresh import conn on primary migration

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      ptlrpc_connection_get() resolves the peer primary once via LNetPrimaryNID() and freezes it in the cached connection. Reconnect reuses that connection and never re-resolves, and ptl_send_rpc() keys the reply ME on the frozen NID. When LNet's primary for the peer later migrates, the reply ME stops matching the source NID LNet stamps on the reply, because lnet_parse() normalizes that source to the current primary. This happens for any peer whose primary we don't lock, i.e. every client peer as seen from a server. Once we're in that state every reply takes the expensive lnet_try_match_md() same-peer fallback (two lnet_find_peer() lookups under lnet_net_lock plus a CWARN), and if the peer table is corrupt the mismatch becomes an outright drop. On a reverse-channel AST a dropped reply means the server evicts the client.

      Add LNetPeerPrimaryNID(), a non-blocking primary lookup that neither runs discovery nor allocates a peer, and call it from ptl_send_rpc(). On a detected migration, get a connection keyed on the current primary and swap imp_connection before the reply ME is posted. ptl_send_rpc() is the chokepoint both client RPCs and server reverse-channel ASTs pass through, so the one hook covers both directions, and it self-heals after a reconnect because the next send re-checks.

            hornc Chris Horn
            hornc Chris Horn
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated: