[LU-1967] 2.2<->2.3 Test failure on test suite recovery-small, subtest test_104

• Type: Bug
• Resolution: Cannot Reproduce
• Priority: Minor
• Environment: server: 2.3 tag-2.2.96; client: 2.2.0
• Severity: 3
• Assignee: WC Triage
• Reporter: Maloo

Description

This issue was created by maloo for sarah <[e-mail]>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/969bd8c2-ff5e-11e1-bce0-52540035b04c.

The sub-test test_104 failed with the following error:

ir status on ost1 should be OFF

== recovery-small test 104: IR: ost can disable IR voluntarily == 03:14:42 (1347704082)
CMD: client-26vm3 lctl set_param -n mgs.MGS.live.lustre=state=full
CMD: client-26vm4 grep -c /mnt/ost1' ' /proc/mounts
Stopping /mnt/ost1 (opts:) on client-26vm4
CMD: client-26vm4 umount -d /mnt/ost1
CMD: client-26vm4 lsmod | grep lnet > /dev/null && lctl dl | grep ' ST '
CMD: client-26vm4 mkdir -p /mnt/ost1
Starting ost1:  -onoir  /dev/lvm-OSS/P1 /mnt/ost1
CMD: client-26vm4 mkdir -p /mnt/ost1; mount -t lustre  -onoir  /dev/lvm-OSS/P1 /mnt/ost1
CMD: client-26vm4 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/lib64/openmpi/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin: NAME=autotest_config sh rpc.sh set_default_debug \"-1\" \" 0xffb7e3ff\" 32 
CMD: client-26vm4 e2label /dev/lvm-OSS/P1
Started lustre-OST0000
CMD: client-26vm4 lctl get_param -n obdfilter.lustre-OST0000.recovery_status |
                               awk '/IR:/{ print \$2}'
 recovery-small test_104: @@@@@@ FAIL: ir status on ost1 should be OFF 

Issue Links

• duplicates: LU-1745 Test failure on test suite recovery-small, subtest test_105

Comments

[Andreas Dilger]

Close old ticket.

[Saurabh Tandan (Inactive)]

Encountered same issue for Failover testing with Client: SLES 11.3 and server: RHEL 6.7
Following is the link of issue : https://testing.hpdd.intel.com/test_sets/b99457d4-632c-11e5-afa7-5254006e85c2

recovery-small test_104: @@@@@@ FAIL: Error state , must be ENABLED or DISABLED 
  Trace dump:
  = /usr/lib64/lustre/tests/test-framework.sh:4784:error_noexit()
  = /usr/lib64/lustre/tests/test-framework.sh:4815:error()
  = /usr/lib64/lustre/tests/recovery-small.sh:1501:check_target_ir_state()
  = /usr/lib64/lustre/tests/recovery-small.sh:1743:test_104()
  = /usr/lib64/lustre/tests/test-framework.sh:5062:run_one()
  = /usr/lib64/lustre/tests/test-framework.sh:5099:run_one_logged()
  = /usr/lib64/lustre/tests/test-framework.sh:4964:run_test()
  = /usr/lib64/lustre/tests/recovery-small.sh:1747:main()
CMD: onyx-42vm1,onyx-42vm3,onyx-42vm5,onyx-42vm6,onyx-42vm8 /usr/sbin/lctl dk > /logdir/test_logs/2015-09-22/lustre-master-el6_7-x86_64-vs-lustre-master-sles11sp3-x86_64--failover--2_1_1__3194__-70064510799500-091051/recovery-small.test_104.debug_log.\$(hostname -s).1443120533.log;
         dmesg > /logdir/test_logs/2015-09-22/lustre-master-el6_7-x86_64-vs-lustre-master-sles11sp3-x86_64--failover--2_1_1__3194__-70064510799500-091051/recovery-small.test_104.dmesg.\$(hostname -s).1443120533.log
/usr/lib64/lustre/tests/recovery-small.sh: line 1744: [: too many arguments
 recovery-small test_104: @@@@@@ FAIL: ir status on ost1 should be DISABLED 

[Jian Yu]

This is a duplicate of LU-1745.

Part of Peter's comment in LU-1912:
We don't have any plans to land anything to b2_2 at this time. This test failure will disappear when we switching to the 2.4 interop matrix.

[Bob Glossman (Inactive)]

I don't think this is fixable in master. commit 6af4ec4c33cc2ea97094c5fab69d9d637d2d82d3 switched the state report in 2.3 from ON/OFF to ENABLED/DISABLED. The test script in 2.3/master was altered to accept either OFF or DISABLED as valid states, but the test script in 2.2 still accepts only OFF. For a 2.2 client on 2.3 or later server the fix needs to be in the 2.2 test script.