[LU-20213] sanity-sec test_{45,47} interop: client bulk callback failed

• Type: Bug
• Resolution: Unresolved
• Priority: Medium
• Severity: 3
• Assignee: Jinshan Xiong
• Reporter: Maloo

Description

This issue was created by maloo for Marc Vef <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/650a2103-5a42-4d6e-b2e4-5f4775067baf

test_47 failed with the following error:

Timeout occurred after 165 minutes, last suite running was sanity-sec

Test session details:
clients: https://build.whamcloud.com/job/lustre-master-next/976 - 4.18.0-553.117.1.el8_10.x86_64
servers: https://build.whamcloud.com/job/lustre-b2_16/6 - 5.14.0-427.31.1_lustre.el9.x86_64

[Fri May  1 13:40:12 2026] Lustre: DEBUG MARKER: grep -c /mnt/lustre' ' /proc/mounts
[Fri May  1 13:40:12 2026] Lustre: DEBUG MARKER: lsof -t /mnt/lustre
[Fri May  1 13:40:12 2026] Lustre: DEBUG MARKER: umount  /mnt/lustre 2>&1
[Fri May  1 13:40:12 2026] Lustre: DEBUG MARKER: mkdir -p /mnt/lustre
[Fri May  1 13:40:12 2026] Lustre: DEBUG MARKER: mount -t lustre -o user_xattr,flock,test_dummy_encryption 10.240.31.209@tcp:/lustre /mnt/lustre
[Fri May  1 13:40:12 2026] LustreError: 267770:0:(events.c:192:client_bulk_callback()) event type 1, status -5, req ff3250c304619a00 desc ff3250c302cda400 mbits 1863993299179136
[Fri May  1 13:40:12 2026] Lustre: 267778:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1777642812/real 1777642812]  req@ff3250c304619a00 x1863993299179136/t0(0) o4->lustre-OST0005-osc-ff3250c305690000@10.240.31.208@tcp:6/4 lens 488/448 e 0 to 1 dl 1777642828 ref 2 fl Rpc:eXQr/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0
[Fri May  1 13:40:12 2026] Lustre: 267778:0:(client.c:2479:ptlrpc_expire_one_request()) Skipped 6 previous similar messages
[Fri May  1 13:40:12 2026] Lustre: lustre-OST0005-osc-ff3250c305690000: Connection to lustre-OST0005 (at 10.240.31.208@tcp) was lost; in progress operations using this service will wait for recovery to complete
[Fri May  1 13:40:12 2026] Lustre: Skipped 6 previous similar messages
[Fri May  1 13:40:13 2026] LNet: 1 local NIs in recovery (showing 1): 10.240.31.206@tcp
[Fri May  1 13:40:17 2026] Lustre: lustre-OST0005-osc-ff3250c305690000: Connection restored to 10.240.31.208@tcp (at 10.240.31.208@tcp)
[Fri May  1 13:40:17 2026] LustreError: 267770:0:(events.c:192:client_bulk_callback()) event type 1, status -5, req ff3250c304619a00 desc ff3250c302cda400 mbits 1863993299180544
[Fri May  1 13:40:17 2026] Lustre: 267778:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1777642817/real 1777642817]  req@ff3250c304619a00 x1863993299179136/t0(0) o4->lustre-OST0005-osc-ff3250c305690000@10.240.31.208@tcp:6/4 lens 488/448 e 0 to 1 dl 1777642833 ref 2 fl Rpc:eXQr/602/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0
[Fri May  1 13:40:17 2026] Lustre: lustre-OST0005-osc-ff3250c305690000: Connection to lustre-OST0005 (at 10.240.31.208@tcp) was lost; in progress operations using this service will wait for recovery to complete
[Fri May  1 13:40:22 2026] Lustre: lustre-OST0005-osc-ff3250c305690000: Connection restored to 10.240.31.208@tcp (at 10.240.31.208@tcp)
[Fri May  1 13:40:22 2026] LustreError: 267770:0:(events.c:192:client_bulk_callback()) event type 1, status -5, req ff3250c304619a00 desc ff3250c302cda400 mbits 1863993299181952
[Fri May  1 13:40:22 2026] Lustre: 267778:0:(client.c:2479:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1777642823/real 1777642823]  req@ff3250c304619a00 x1863993299179136/t0(0) o4->lustre-OST0005-osc-ff3250c305690000@10.240.31.208@tcp:6/4 lens 488/448 e 0 to 1 dl 1777642839 ref 2 fl Rpc:eXQr/602/ffffffff rc 0/-1  

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity-sec test_47 - Timeout occurred after 165 minutes, last suite running was sanity-sec

Issue Links

• is related to: LU-19763 Enable TCP zerocopy sending by default