[LU-10002] client_bulk_callback() event type 1, status -5, desc ffff881fe5a8f800

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Labels: llnl
• Environment: ssh://review.whamcloud.com/fs/lustre-release-fe-llnl; Client is lustre-2.8.0_11.chaos; Server is lustre-2.5.5-13chaos; both x86_64
• Severity: 3
• Assignee: Sonia Sharma (Inactive)
• Reporter: Olaf Faaland

Description

Repeated console log entries like this on a client while running mdtest:

Lustre: lcy-OST000c-osc-ffff883ff7d0e800: Connection restored to 10.1.1.183@o2ib9 (at 10.1.1.183@o2ib9)
Lustre: Skipped 17 previous similar messages
mlx5_0:dump_cqe:262:(pid 5096): dump error cqe
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 08007806 25000230 001344d2
LustreError: 5099:0:(events.c:203:client_bulk_callback()) event type 1, status -5, desc ffff881fe5a8b800
LustreError: 5098:0:(events.c:203:client_bulk_callback()) event type 1, status -5, desc ffff881fd8390800
mlx5_0:dump_cqe:262:(pid 5097): dump error cqe
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 08007806 25000231 00044ad2

Along with reconnects to the servers.

This issue occurs when running with the patches backported under LU-9932

Comments

[Peter Jones]

Seems that this issue can be closed

[Olaf Faaland]

That appears to have solved the issue. Thanks.

[Olaf Faaland]

Hi Sonia,
Yes we have nodes with mlx5 cards connecting to nodes with mlx4 cards, and no we do not have LU-8752. We will get that patch added.
Thanks

[Sonia Sharma (Inactive)]

Hi Olaf,
Do you have nodes with mlx5 cards talking to nodes with mlx4 cards in your setup?
The dump cqe error is seen on the nodes with mlx5 cards. You would need LU-8752 patch on the nodes with mlx5 cards. Can you check if LU-8752 is included on these nodes?

Thanks

[Peter Jones]

Sonia

Can you please advise

Thanks

Peter

[Olaf Faaland]

I see the dump_cqe message is comgin from the infiniband driver