Jan 21 17:06:40 slurm-vm-1 systemd-logind: New session 1 of user centos.
Jan 21 17:06:40 slurm-vm-1 systemd: Started Session 1 of user centos.
Jan 21 17:06:40 slurm-vm-1 systemd: Starting Session 1 of user centos.
Jan 21 17:13:32 slurm-vm-1 systemd: Started Session 2 of user centos.
Jan 21 17:13:32 slurm-vm-1 systemd-logind: New session 2 of user centos.
Jan 21 17:13:32 slurm-vm-1 systemd: Starting Session 2 of user centos.
Jan 21 17:15:14 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1854): dump error cqe
Jan 21 17:15:14 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:14 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:14 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:14 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 6c 64 26 d0 d2
Jan 21 17:15:14 slurm-vm-1 kernel: LustreError: 1854:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:14 slurm-vm-1 kernel: Lustre: 1860:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1548062114
/real 1548062114]  req@ffff8bafb29ce400 x1623260079965728/t0(0) o4->hpcfs-OST0006-osc-ffff8baffda05000@172.25.0.213@o2ib1:6/4 lens 608/448 e 0 to 1 dl 154806212
1 ref 2 fl Rpc:eX/0/ffffffff rc 0/-1
Jan 21 17:15:14 slurm-vm-1 kernel: Lustre: hpcfs-OST0006-osc-ffff8baffda05000: Connection to hpcfs-OST0006 (at 172.25.0.213@o2ib1) was lost; in progress operati
ons using this service will wait for recovery to complete
Jan 21 17:15:14 slurm-vm-1 kernel: Lustre: hpcfs-OST0006-osc-ffff8baffda05000: Connection restored to 172.25.255.213@o2ib (at 172.25.0.213@o2ib1)
Jan 21 17:15:14 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1850): dump error cqe
Jan 21 17:15:14 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:14 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:14 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:14 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 6b 0f ef d6 d2
Jan 21 17:15:14 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:14 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2b70e00
Jan 21 17:15:14 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2b71600
Jan 21 17:15:14 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2b71000
Jan 21 17:15:14 slurm-vm-1 kernel: LustreError: 1864:0:(import.c:671:ptlrpc_connect_import()) already connecting
Jan 21 17:15:21 slurm-vm-1 kernel: Lustre: 1860:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1548062114/re
al 1548062114]  req@ffff8bafb2c74300 x1623260079966048/t0(0) o4->hpcfs-OST0006-osc-ffff8baffda05000@172.25.0.213@o2ib1:6/4 lens 608/448 e 0 to 1 dl 1548062121 r
ef 2 fl Rpc:X/2/ffffffff rc -11/-1
Jan 21 17:15:21 slurm-vm-1 kernel: Lustre: 1860:0:(client.c:2114:ptlrpc_expire_one_request()) Skipped 5 previous similar messages
Jan 21 17:15:39 slurm-vm-1 kernel: Lustre: hpcfs-OST0006-osc-ffff8baffda05000: Connection restored to 172.25.255.213@o2ib (at 172.25.0.213@o2ib1)
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1850): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 6e 00 6f 5b d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2f68a00
Jan 21 17:15:39 slurm-vm-1 kernel: Lustre: 1860:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1548062139
/real 1548062139]  req@ffff8bafb29ce400 x1623260079965728/t0(0) o4->hpcfs-OST0006-osc-ffff8baffda05000@172.25.0.213@o2ib1:6/4 lens 608/448 e 0 to 1 dl 154806214
6 ref 2 fl Rpc:eX/2/ffffffff rc 0/-1
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2b71400
Jan 21 17:15:39 slurm-vm-1 kernel: Lustre: hpcfs-OST0006-osc-ffff8baffda05000: Connection to hpcfs-OST0006 (at 172.25.0.213@o2ib1) was lost; in progress operati
ons using this service will wait for recovery to complete
Jan 21 17:15:39 slurm-vm-1 kernel: Lustre: Skipped 1 previous similar message
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2b71600
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2b70e00
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1854): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 6d 0a 8b fa d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1854:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1854:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2b70e00
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1855:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafe967aa00
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1855): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 70 02 63 7b d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1855:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1854:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafe9679600
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1854): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 71 00 5f 5b d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1854:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 6f 16 a8 2a d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 73 00 5d 5f d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 74 00 5d 58 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 75 00 5d 59 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 76 00 5d 5a d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 77 00 5d 5b d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 78 00 5d 54 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 79 00 5d 55 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 7a 00 5d 56 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 7b 00 5d 57 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 7c 00 5d 50 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 7d 00 5d 51 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 7e 00 5d 52 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 7f 00 5d 53 d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 80 00 5d ac d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1851:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 81 00 5d ad d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400
Jan 21 17:15:39 slurm-vm-1 kernel: mlx5_0:dump_cqe:285:(pid 1851): dump error cqe
Jan 21 17:15:39 slurm-vm-1 kernel: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Jan 21 17:15:39 slurm-vm-1 kernel: 00000030: 00 00 00 00 02 00 51 04 08 00 00 82 00 5d ae d2
Jan 21 17:15:39 slurm-vm-1 kernel: LustreError: 1850:0:(events.c:199:client_bulk_callback()) event type 1, status -5, desc ffff8bafb2976400