[LU-17325] o2iblnd: graceful handling of CM_EVENT_UNREACHABLE on established connection

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0
• Labels: RoCE o2iblnd
• Severity: 3
• Assignee: Serguei Smirnov
• Reporter: Serguei Smirnov

Description

There were examples in the field with RoCE setups which demonstrate that CM_EVENT_UNREACHABLE may be received when connection is already in ESTABLISHED state.

This causes the assertion in kiblnd_cm_callback() to fail:

 ASSERTION( conn->ibc_state != 3 && conn->ibc_state != 0 ) failed:

It is proposed to handle this in a more gracious manner:  report the event as unexpected and allow the flow to continue. If there are indeed issues on the connection, it is expected to report transaction errors and get cleaned up without crashing the whole system.