-
Bug
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
-
3
-
9223372036854775807
sanity-lnet test_237 failed with "Failed to ping 10.240.26.110@tcp1" (-113) on b_es7_0:
https://testing.whamcloud.com/test_sets/eb812b2b-d1a4-4982-8eeb-3e8ffd79e973
The route to tcp1 went down because a discovery Push from the client to the router failed. The Push used a source NID and a destination NID on different local networks:
lnet_handle_send_case_locked()) Source Specified: 10.240.26.107@tcp2 to MR: 10.240.26.109@tcp routed destination lnet_handle_find_routed_path()) Unable to send message from 10.240.26.107@tcp2 to 10.240.26.109@tcp - Route table may be misconfigured LNetPut()) Error sending PUT to 12345-10.240.26.109@tcp: -113 lnet_router_discovery_complete()) 10.240.26.109@tcp: Router discovery failed -113 lnet_set_route_aliveness()) route to tcp1 through 10.240.26.109@tcp has gone from up to down
The client debug log shows this sequence:
- The discovery thread sends a ping to 10.240.26.109@tcp. LNet sends it from 10.240.26.107@tcp2 to 10.240.26.109@tcp2.
- Before the reply arrives, a Push from the router arrives. The discovery thread merges the Push data and calls lnet_peer_send_push().
- lnet_peer_send_push() reads lp_disc_dst_nid. The value is LNET_ANY_NID, so the destination is the primary NID 10.240.26.109@tcp.
- The ping reply arrives. lnet_discovery_event_reply() sets lp_disc_src_nid to 10.240.26.107@tcp2 and lp_disc_dst_nid to 10.240.26.109@tcp2.
- LNetPut() reads lp->lp_disc_src_nid through a pointer and gets 10.240.26.107@tcp2.
lnet_peer_send_push() drops lp_lock before it reads lp_disc_dst_nid and lp_disc_src_nid. lnet_discovery_event_reply() writes both fields under lp_lock. The two reads can see different replies. The same code is on master.
- resolves
-
EX-16264 Loading...