Uploaded image for project: 'Lustre'
  1. Lustre
  2. LU-20482

Secondary-NID renumbering on live peer trips BAD_CONFIG guard and hangs I/O

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Medium Medium
    • None
    • None
    • None
    • 3
    • 9223372036854775807

      Summary:

      When a multi-rail peer's secondary NID is renumbered while
      its primary NID remains reachable, LNetAddPeer() builds a temporary
      peer for the new NID and relies on discovery to merge it into the
      existing peer. In lnet_peer_add_nid() the merge collides with a safety
      check when the existing peer has LNET_PEER_LOCK_PRIMARY set
      with more than one NI. Both peers are marked LNET_PEER_BAD_CONFIG and
      the client hangs on all I/O to that server.
       

      Scenario:

      Server peer state on the client transitions:
       
      [TCP_NID, OLD_NID] --> [TCP_NID, NEW_NID] 
       --> means the server rebooted directly. No lnetctl peer del and no
      client-side unmount happens in between.
      On reconnection the client learns the new NID list and
      calls LNetAddPeer([TCP_NID, NEW_NID]). TCP_NID is the primary and remains
      reachable; OLD_NID was a secondary on the same peer and has been replaced
      by NEW_NID.
       
      LNetAddPeer() builds a fresh peer lp with lp_primary_nid = NEW_NID
      and lp_merge_primary_nid = TCP_NID, and kicks discovery on NEW_NID.
      Discovery's ping reply lists [TCP_NID, NEW_NID]. lnet_peer_merge_data(lp, pbuf)
      computes addnis = [TCP_NID] and calls lnet_peer_add_nid(lp, TCP_NID, ...).
      This will fail because TCP_NID already has it's peer [TCP_NID, OLD_NID].
       
      Would like to know if this is a valid concern, and if so if the proposed fix makes sense.

            wc-triage WC Triage
            zanhua Zanhua Huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: