[LU-12901] Failing to create a properly sized IB queue pair

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.14.0
• Affects Version/s: Lustre 2.13.0
• Environment: Seen with newer Mellanox ConnectX-4 devices
• Severity: 3
• Assignee: Serguei Smirnov
• Reporter: James A Simmons

Description

Attempting to bring up a file system in our test bed with the latest lustre version (2.13) I saw this new error on LNet bring up.

[ 472.738363] LNet: 8481:0:(o2iblnd_cb.c:3395:kiblnd_check_conns()) Timed out tx for 10.37.248.232@o2ib1: 471 seconds
[ 473.739295] LNetError: 2014:0:(o2iblnd.c:929:kiblnd_create_conn()) Can't create QP: -12, send_wr: 16317, recv_wr: 128, send_sge: 2, recv_sge: 1

I found I can lower the peer_credits to get around this but that is not the proper fix.

 

 

 

 

Issue Links

• is related to: LU-10213 o2iblnd: Potential discrepancy when allocating qp
• is related to: LU-7124 MLX5: Limit hit in cap.max_send_wr