[LU-8333] replay-dual test_21b: can't check if COS works: rename replied w/o COS

• Type: Bug
• Resolution: Unresolved
• Priority: Critical
• Affects Version/s: Lustre 2.7.0, Lustre 2.9.0, Lustre 2.10.0, Lustre 2.11.0
• Labels: always_except
• Environment: Hard failover EL7 Server/Client; master, build# 3399
• Severity: 3
• Assignee: Hongchao Zhang
• Reporter: Maloo

Description

This issue was created by maloo for Saurabh Tandan <[e-mail]>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/31581902-3ad4-11e6-bbf5-5254006e85c2.

The sub-test test_21b failed with the following error:

can't check if COS works: rename replied w/o COS

Test log:

Started clients trevis-56vm5: 
CMD: trevis-56vm5 mount | grep /mnt/lustre' '
10.9.6.124@tcp:10.9.6.120@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr)
CMD: trevis-56vm5 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/lib64/qt-3.3/bin:/usr/lib64/compat-openmpi16/bin:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh set_default_debug \"-1\" \"all -lnet -lnd -pinger\" 4 
 replay-dual test_21b: @@@@@@ FAIL: can't check if COS works: rename replied w/o COS 

In past 15 days this issue has occurred around 36 times.

Issue Links

• is duplicated by: LU-2230 replay-dual test_21b: @@@@@@ FAIL: The test cannot check whether COS works or not: all renames are replied w/o COS
• is duplicated by: LU-4104 Failure on test suite replay-dual test_21b: Not all renames are replayed. COS=1
• is related to: LU-9586 replay-dual test cases 15c 21b remove from ALWAYS_EXCEPT
• is related to: LU-4470 replay-dual test_21b: FAIL: lustre-MDT0000: ffff88007f581100 CANNOT BE SET READONLY: rc = -95

Comments

[Joseph Gmitter (Inactive)]

Hi Mike,

Can you comment on the validity of this test case in relation to commit on share? Is it a valid test case, providing useful testing?

Thanks.
Joe

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/25848
Subject: LU-8333 test: disable replay-dual 21b
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 9d046935201bf2ea6b0e788f16c2d1bdaf843577

[Hongchao Zhang]

I have checked the recent failures and found the transactions of the client2 (intended to introduce the transaction gap)
has been committed before failing over the MDT, which cause the client1 to recover successfully.

I'll consider some new means to test the COS feature.

[Gerrit Updater]

Hongchao Zhang ([e-mail]) uploaded a new patch: http://review.whamcloud.com/23830
Subject: LU-8333 test: debug log
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 4df7fb623fb09fe078d86d51361e1f39e2db792d

[Saurabh Tandan (Inactive)]

Reopening this ticket as the issue is still seen on master consistently. Please refer previous message above for latest failures.

[Saurabh Tandan (Inactive)]

This issue is still seen on master regularly.
https://testing.hpdd.intel.com/test_sets/c4a7f648-a1c0-11e6-8ed2-5254006e85c2
https://testing.hpdd.intel.com/sub_tests/4cab1870-a2d6-11e6-8986-5254006e85c2
https://testing.hpdd.intel.com/sub_tests/ca5e0b42-a2ae-11e6-8b77-5254006e85c2
https://testing.hpdd.intel.com/sub_tests/05adec2a-a274-11e6-8986-5254006e85c2

[Peter Jones]

Landed for 2.9

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch http://review.whamcloud.com/21924/
Subject: LU-8333 test: make sure COS is cleared
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: ae2ccbc6da32def014463ee98fab76a93835d85d

[Gerrit Updater]

Hongchao Zhang ([e-mail]) uploaded a new patch: http://review.whamcloud.com/21924
Subject: LU-8333 test: make sure COS is cleared
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: d0f614f780182e3ea56290e215cf5da73faf2cf1

[Peter Jones]

Hongchao

Could you please advise on this one?

Thanks

Peter