-
Bug
-
Resolution: Unresolved
-
Medium
-
None
-
None
-
None
-
3
-
9223372036854775807
Problem
o2iblnd can call kiblnd_tx_complete() on a PUT_DONE tx with a NULL tx_conn. The node then crashes with a NULL pointer dereference. The crash needs a conn that finalises while the scheduler handles a PUT_ACK.
Environment
- master at 4a7bddaf15, and each branch that has a155c3fca38d ("
LU-11756o2iblnd: kib_conn leak"). - o2ib with any HCA. The window is wider when a device removal or a reset closes conns under traffic.
Reproduction
We have no reproducer. We found the path in the source during a review of the LU-6132 series.
Analysis
- o2iblnd posts a PUT_REQ tx.
- The PUT_ACK arrives before the PUT_REQ send completes. kiblnd_handle_rx() removes the tx from ibc_active_txs. tx_sending is still 1.
- connd finalises the conn and sets IBLND_CONN_DISCONNECTED. kiblnd_abort_txs() does not see the tx, because the tx is on no list.
- kiblnd_handle_rx() calls kiblnd_queue_tx_locked() for the PUT_DONE. The DISCONNECTED branch puts the tx on ibc_zombie_txs, clears tx_conn, and drops the conn ref.
- The HCA flushes the PUT_REQ send. kiblnd_tx_complete() reads tx->tx_conn, which is NULL.
If the HCA does not flush, no code completes the posted send. kiblnd_destroy_conn() then hits LASSERT(conn->ibc_nsends_posted == 0) when the last conn ref goes.
Proposed fix
In the DISCONNECTED branch of kiblnd_queue_tx_locked(), keep a PUT_DONE with tx_sending != 0 on ibc_active_txs with its conn ref. The send completion then releases the tx through the normal idle path.
- is related to
-
LU-6132 Unable to unload ib drivers with lustre loaded
-
- In Progress
-