Whamcloud Community JIRA https://jira.whamcloud.com This file is an XML representation of an issue en-us 9.4.14 940014 05-12-2023 [LU-13821] Lustre: 2835:0:(client.c:2133:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: https://jira.whamcloud.com/browse/LU-13821 Lustre <p>Clients experience timeouts when running multiple rsyncs on them.  </p> <p> </p> <p>On the client - </p> <p>Jul 25 08:57:30 zabbix01 kernel: Lustre: 2812:0:(client.c:2133:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: <span class="error">&#91;sent 1595681806/real 1595681806&#93;</span> req@ffff8921e2bcda00 x1673155962380928/t0(0) o36-&gt;lustre01-MDT0000-mdc-ffff891ff5c7b800@10.42.34.30@tcp:12/10 lens 488/4528 e 0 to 1 dl 1595681850 ref 2 fl Rpc:X/0/ffffffff rc 0/-1<br/> Jul 25 08:57:30 zabbix01 kernel: Lustre: lustre01-MDT0000-mdc-ffff891ff5c7b800: Connection to lustre01-MDT0000 (at 10.42.34.30@tcp) was lost; in progress operations using this service will wait for recovery to complete<br/> Jul 25 08:57:30 zabbix01 kernel: Lustre: lustre01-MDT0000-mdc-ffff891ff5c7b800: Connection restored to 10.42.34.30@tcp (at 10.42.34.30@tcp)</p> <p> </p> <p> </p> <p>On the server - </p> <p>Jul 25 08:57:30 lustremds01 kernel: Lustre: lustre01-MDT0000: Client 8e8cc5cc-b257-0497-a475-10e92f1051df (at 130.199.148.189@tcp) reconnecting<br/> Jul 25 08:57:30 lustremds01 kernel: Lustre: lustre01-MDT0000: Connection restored to 5467a4ae-f9e4-bdcf-f38b-0f32f0db3a8d (at 130.199.148.189@tcp)</p> <p> </p> <p>Also attaching "lctl dk" output for both server/client</p> RHEL 7.8 LU-13821 Lustre: 2835:0:(client.c:2133:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: Bug Critical Open Unresolved WC Triage Joe Frith Sat, 25 Jul 2020 13:15:29 +0000 Wed, 7 Oct 2020 17:40:57 +0000 Lustre 2.12.5 0 3 <p>Occasionally I also see the below on the server (MDS) side. </p> <p> </p> <p>Jul 25 09:03:07 lustremds01 kernel: LustreError: 12107:0:(ldlm_lib.c:3279:target_bulk_io()) @@@ Reconnect on bulk READ req@ffff98064af38050 x1673156574796480/t0(0) o37-&gt;8e8cc5cc-b257-0497-a475-10e92f1051df@130.199.148.189@tcp:263/0 lens 448/440 e 0 to 0 dl 1595682193 ref 1 fl Interpret:/0/0 rc 0/0</p> <p>Is there a tunable I can change to avoid this? This is causing high latency and seems to happen often now. </p> <p>Any update?</p> <p>Hello, any update on this? </p> Related Development Rank 1|i0166v: Rank (Obsolete) 9223372036854775807 Severity