[LU-9124] replay-single test_70c test failed to respond and timed out

• Type: Bug
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.10.0
• Environment: onyx-30vm1-3/7/8, Full Group test, ; master branch, v2.9.52, b3520, ; DNE, ZFS
• Severity: 3
• Assignee: WC Triage
• Reporter: James Casper

Description

https://testing.hpdd.intel.com/test_sets/2dffea4e-efe9-11e6-8c0d-5254006e85c2

Related to closed tickets LU-8107 and LU-6844.

Noticed the following in the client 1 console log:

18:06:08:[ 5809.688144] Lustre: lustre-MDT0000-mdc-ffff88007afe0000: Connection to lustre-MDT0000 (at 10.2.4.99@tcp) was lost; in progress operations using this service will wait for recovery to complete
18:06:08:[ 5809.693121] Lustre: Skipped 9 previous similar messages
18:06:08:[ 5818.342043] Lustre: 6225:0:(client.c:2113:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1486602626/real 1486602626]  req@ffff880056b20900 x1558809913645904/t0(0) o400->MGC10.2.4.99@tcp@10.2.4.99@tcp:26/25 lens 224/224 e 0 to 1 dl 1486602633 ref 1 fl Rpc:XN/0/ffffffff rc 0/-1
18:06:08:[ 5818.349284] Lustre: 6225:0:(client.c:2113:ptlrpc_expire_one_request()) Skipped 5 previous similar messages
18:06:08:[ 5845.040458] Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/u
18:06:08:[ 5845.359219] Lustre: DEBUG MARKER: lctl get_param -n at_max
18:06:08:[ 5845.553830] LustreError: 6223:0:(client.c:2981:ptlrpc_replay_interpret()) @@@ status 301, old was 0  req@ffff880049055b00 x1558809913642944/t317827580321(317827580321) o101->lustre-MDT0000-mdc-ffff88007afe0000@10.2.4.99@tcp:12/10 lens 824/544 e 0 to 0 dl 1486602701 ref 2 fl Interpret:R/4/0 rc 301/301
18:06:08:[ 5845.561253] LustreError: 6223:0:(client.c:2981:ptlrpc_replay_interpret()) Skipped 1 previous similar message