[LU-3049] Interop 1.8.9<->2.4 failure on test suite replay-vbr test_0b

• Status: Resolved
• Project: Lustre
• Affects Version/s: Lustre 2.4.0, Lustre 2.4.1
• Type: Bug
• Priority: Major
• Reporter: Maloo
• Assignee: Mikhail Pershin
• Resolution: Cannot Reproduce
• Labels: yuc2

Environment

client: 1.8.9
server: lustre-master build #1338

• Severity: 3

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/277e3fa0-948e-11e2-93c6-52540035b04c.

The sub-test test_0b failed with the following error:

client-27vm6.lab.whamcloud.com not up

Lustre: DEBUG MARKER: == replay-vbr test 0b: VBR: open (O_CREAT) checks version of parent == 06:57:22 (1364047042)
Lustre: DEBUG MARKER: mkdir -p -m 755 /mnt/lustre/d0.replay-vbr/d0
Lustre: DEBUG MARKER: local REPLAY BARRIER on lustre-MDT0000
Lustre: DEBUG MARKER: openfile -f O_RDWR:O_CREAT /mnt/lustre/d0.replay-vbr/d0/f0b
Lustre: 8536:0:(import.c:855:ptlrpc_connect_interpret()) MGS@10.10.4.164@tcp changed server handle from 0x1ed7ccc40babfa14 to 0x1ed7ccc40bac0137
Lustre: 8536:0:(client.c:2359:ptlrpc_replay_interpret()) @@@ Version mismatch during replay
  req@ffff88006ae91c00 x1430302317102617/t128849018903 o101->lustre-MDT0000_UUID@10.10.4.164@tcp:12/10 lens 688/1456 e 1 to 1 dl 1364047216 ref 2 fl Interpret:R/4/0 rc -75/-75
LustreError: 8536:0:(client.c:2391:ptlrpc_replay_interpret()) @@@ status -75, old was 301  req@ffff88006ae91c00 x1430302317102617/t128849018903 o101->lustre-MDT0000_UUID@10.10.4.164@tcp:12/10 lens 688/1456 e 1 to 1 dl 1364047216 ref 2 fl Interpret:R/4/0 rc -75/-75
Lustre: 8536:0:(client.c:1529:ptlrpc_expire_one_request()) @@@ Request x1430302317102800 sent from lustre-MDT0000-mdc-ffff88007acae800 to NID 10.10.4.164@tcp 85s ago has timed out (85s prior to deadline).
  req@ffff88007c9e6000 x1430302317102800/t0 o400->lustre-MDT0000_UUID@10.10.4.164@tcp:12/10 lens 192/384 e 0 to 1 dl 1364047231 ref 1 fl Rpc:/c1/0 rc 0/0
Lustre: 8536:0:(client.c:1529:ptlrpc_expire_one_request()) Skipped 8 previous similar messages
Lustre: 8536:0:(import.c:1160:completed_replay_interpret()) lustre-MDT0000-mdc-ffff88007acae800: version recovery fails, reconnecting
LustreError: 167-0: This client was evicted by lustre-MDT0000; in progress operations using this service will fail.
LustreError: 20621:0:(llite_lib.c:1778:ll_statfs_internal()) mdc_statfs fails: rc = -4
Lustre: DEBUG MARKER: PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/tests:/usr/lib64/openmpi/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/lib64
Lustre: DEBUG MARKER: /usr/sbin/lctl mark  replay-vbr test_0b: @@@@@@ FAIL: import is not in FULL state 
Lustre: DEBUG MARKER: replay-vbr test_0b: @@@@@@ FAIL: import is not in FULL state
Lustre: DEBUG MARKER: /usr/sbin/lctl dk > /logdir/test_logs/2013-03-22/lustre-master-el6-x86_64-vs-lustre-b1_8-el6-x86_64--full--2_4_1__1338__-70011888563720-182230/replay-vbr.test_0b.debug_log.$(hostname -s).1364047231.log;
         dmesg > /logdir/test_logs/2013-03-22/lustre

Comments

[Andreas Dilger]

Mike, is replay-vbr expected to work between 1.8 and 2.4?

[Jian Yu]

Lustre client: http://build.whamcloud.com/job/lustre-b1_8/258/ (1.8.9-wc1)
Lustre server: http://build.whamcloud.com/job/lustre-b2_4/45/ (2.4.1 RC2)

replay-vbr test 0b failed again:
https://maloo.whamcloud.com/test_sets/f3c3142e-19ee-11e3-a95b-52540035b04c

[Mikhail Pershin]

replay-vbr.sh works between 1.8 and 2.4 and as I can see tests run as expected, but test script is unable to check import state is FULL after client reconnection. I am not sure why, but it doesn't wait actually for import state but exit immediately, so I suppose this is test script problem, particularly wait_clients_import_state() function.

[Andreas Dilger]

Close old bug that has not been seen in a long time.