[LU-15385] Resending request on EINPROGRESS

Thu, 23 Dec 2021 17:28:23 +0000

We are seeing EINPROGRESS errors on the clients. And some threads deadlock not he client. I am not sure if they are related.
Evicting the client cleans up the deadlock threads.

HUNG THREAD

('5851', 'usm3d.productio')
[<ffffffffc154586f>] osc_io_setattr_end+0xaf/0x280 [osc]
[<ffffffffc0f4cd78>] cl_io_end+0x58/0x140 [obdclass]
[<ffffffffc14defbb>] lov_io_end_wrapper+0xcb/0xd0 [lov]
[<ffffffffc14df3f7>] lov_io_call.isra.33+0x77/0x120 [lov]
[<ffffffffc14df4d2>] lov_io_end+0x32/0xa0 [lov]
[<ffffffffc0f4cd78>] cl_io_end+0x58/0x140 [obdclass]
[<ffffffffc0f4f40f>] cl_io_loop+0x9f/0x1d0 [obdclass]
[<ffffffffc16227d1>] cl_setattr_ost+0x221/0x380 [lustre]
[<ffffffffc15fdcc8>] ll_setattr_raw+0xa18/0xfa0 [lustre]
[<ffffffffa8289a4c>] notify_change+0x26c/0x440
[<ffffffffa826625e>] do_truncate+0x5e/0x90
[<ffffffffa826657a>] do_sys_ftruncate.constprop.12+0xea/0xf0
[<ffffffffa8004954>] do_syscall_64+0x74/0x160
[<ffffffffa88000b6>] entry_SYSCALL_64_after_hwframe+0x59/0xbe
[<ffffffffffffffff>] 0xffffffffffffffff


Fri Dec 17 12:57:54 2021] LustreError: 3382:0:(client.c:1440:after_reply()) Skipped 14 previous similar messages
[Fri Dec 17 13:10:15 2021] LustreError: 3382:0:(client.c:1440:after_reply()) @@@ Resending request on EINPROGRESS  req@ffff9e532fcf2480 x1719326682210048/t0(0) o10->nbp17-OST000f-osc-ffff9e56ce3b9800@10.151.27.149@o2ib:6/4 lens 440/400 e 0 to 0 dl 1639776037 ref 1 fl Rpc:R/2/0 rc 0/-115
[Fri Dec 17 13:10:15 2021] LustreError: 3382:0:(client.c:1440:after_reply()) Skipped 14 previous similar messages
[Fri Dec 17 13:22:42 2021] LustreError: 3382:0:(client.c:1440:after_reply()) @@@ Resending request on EINPROGRESS  req@ffff9e551ea04040 x1719326682299840/t0(0) o10->nbp17-OST0008-osc-ffff9e56ce3b9800@10.151.27.146@o2ib:6/4 lens 440/400 e 0 to 0 dl 1639776785 ref 1 fl Rpc:R/2/0 rc 0/-115
[Fri Dec 17 13:22:42 2021] LustreError: 3382:0:(client.c:1440:after_reply()) Skipped 14 previous similar messages
[Fri Dec 17 13:35:20 2021] LustreError: 3382:0:(client.c:1440:after_reply()) @@@ Resending request on EINPROGRESS  req@ffff9e532fcf2480 x1719326682389376/t0(0) o10->nbp17-OST000f-osc-ffff9e56ce3b9800@10.151.27.149@o2ib:6/4 lens 440/400 e 0 to 0 dl 1639777543 ref 1 fl Rpc:R/2/0 rc 0/-115
[Fri Dec 17 13:35:20 2021] LustreError: 3382:0:(client.c:1440:after_reply()) Skipped 14 previous similar messages

Whamcloud Community JIRA

[LU-15385] Resending request on EINPROGRESS