[LU-1966] Test failure on test suite replay-ost-single, subtest test_6: Destroys weren't done in 5 sec

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Affects Version/s: Lustre 2.3.0
• Severity: 3
• Assignee: Zhenyu Xu
• Reporter: Maloo

Description

This issue was created by maloo for yujian <[e-mail]>

This issue relates to the following test suite run: https://maloo.whamcloud.com/test_sets/0d43f964-fff5-11e1-9f3c-52540035b04c.

The sub-test test_6 failed with the following error:

== replay-ost-single test 6: Fail OST before obd_destroy == 19:17:36 (1347761856)
Waiting for orphan cleanup...
CMD: client-28vm4 /usr/sbin/lctl get_param -n obdfilter.*.mds_sync
Waiting for destroy to be done...
Waiting 0 secs for destroys to be done.
Waiting 1 secs for destroys to be done.
Waiting 2 secs for destroys to be done.
Waiting 3 secs for destroys to be done.
Waiting 4 secs for destroys to be done.
Destroys weren't done in 5 sec.
 replay-ost-single test_6: @@@@@@ FAIL: test_6 failed with 5

Info required for matching: replay-ost-single 6

Issue Links

• duplicates: LU-2620 Failure on test suite replay-ost-single test_6: test_6 failed with 1

Comments

[Jian Yu]

Lustre Tag: v2_3_0_RC2
Lustre Build: http://build.whamcloud.com/job/lustre-b2_3/32
Distro/Arch: RHEL6.3/x86_64
Test Group: failover

The same issue occurred: https://maloo.whamcloud.com/test_sets/c1790860-1509-11e2-9adb-52540035b04c

[Peter Jones]

I think that there is enough evidence to suggest that this is purely a testing issue. We should still continue to work on resolving this and include a fix for an RC2 if one is ready and we need one, but it would not warrant holding the release on its own merits

[Jian Yu]

Lustre Build: http://build.whamcloud.com/job/lustre-b2_3/23
FAILURE_MODE=HARD

autotest run failed: https://maloo.whamcloud.com/test_sets/b5c9fc00-0694-11e2-9b17-52540035b04c
manual run passed: https://maloo.whamcloud.com/test_sets/52192992-0716-11e2-ac99-52540035b04c

[Zhenyu Xu]

I checked the patch landed between 09/07 and 09/13, I think nothing about OSS change during that period could cause this issue, I tend to think it's test issue.

[Jian Yu]

Here is a link to the historical Maloo reports for replay-ost-single test 6 in failover test group:
http://tinyurl.com/8s62nfj

As we can see, the last successful run for this test is on 2012-09-07, and the test has been occurring since 2012-09-13.

[Zhenyu Xu]

http://review.whamcloud.com/4067 just for issue reproduction.