[LU-13821] Lustre: 2835:0:(client.c:2133:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply:

• Type: Bug
• Resolution: Unresolved
• Priority: Critical
• Affects Version/s: Lustre 2.12.5
• Environment: RHEL 7.8
• Severity: 3
• Assignee: WC Triage
• Reporter: Joe Frith

Description

Clients experience timeouts when running multiple rsyncs on them.  

 

On the client - 

Jul 25 08:57:30 zabbix01 kernel: Lustre: 2812:0:(client.c:2133:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1595681806/real 1595681806] req@ffff8921e2bcda00 x1673155962380928/t0(0) o36->lustre01-MDT0000-mdc-ffff891ff5c7b800@10.42.34.30@tcp:12/10 lens 488/4528 e 0 to 1 dl 1595681850 ref 2 fl Rpc:X/0/ffffffff rc 0/-1
Jul 25 08:57:30 zabbix01 kernel: Lustre: lustre01-MDT0000-mdc-ffff891ff5c7b800: Connection to lustre01-MDT0000 (at 10.42.34.30@tcp) was lost; in progress operations using this service will wait for recovery to complete
Jul 25 08:57:30 zabbix01 kernel: Lustre: lustre01-MDT0000-mdc-ffff891ff5c7b800: Connection restored to 10.42.34.30@tcp (at 10.42.34.30@tcp)

 

 

On the server - 

Jul 25 08:57:30 lustremds01 kernel: Lustre: lustre01-MDT0000: Client 8e8cc5cc-b257-0497-a475-10e92f1051df (at 130.199.148.189@tcp) reconnecting
Jul 25 08:57:30 lustremds01 kernel: Lustre: lustre01-MDT0000: Connection restored to 5467a4ae-f9e4-bdcf-f38b-0f32f0db3a8d (at 130.199.148.189@tcp)

 

Also attaching "lctl dk" output for both server/client