[LU-8333] replay-dual test_21b: can't check if COS works: rename replied w/o COS

• Type: Bug
• Resolution: Unresolved
• Priority: Critical
• Affects Version/s: Lustre 2.7.0, Lustre 2.9.0, Lustre 2.10.0, Lustre 2.11.0
• Labels: always_except
• Environment: Hard failover EL7 Server/Client; master, build# 3399
• Severity: 3
• Assignee: Hongchao Zhang
• Reporter: Maloo

Description

This issue was created by maloo for Saurabh Tandan <[e-mail]>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/31581902-3ad4-11e6-bbf5-5254006e85c2.

The sub-test test_21b failed with the following error:

can't check if COS works: rename replied w/o COS

Test log:

Started clients trevis-56vm5: 
CMD: trevis-56vm5 mount | grep /mnt/lustre' '
10.9.6.124@tcp:10.9.6.120@tcp:/lustre on /mnt/lustre type lustre (rw,flock,user_xattr)
CMD: trevis-56vm5 PATH=/usr/lib64/lustre/tests:/usr/lib/lustre/tests:/usr/lib64/lustre/tests:/opt/iozone/bin:/opt/iozone/bin:/usr/lib64/lustre/tests/mpi:/usr/lib64/lustre/tests/racer:/usr/lib64/lustre/../lustre-iokit/sgpdd-survey:/usr/lib64/lustre/tests:/usr/lib64/lustre/utils/gss:/usr/lib64/lustre/utils:/usr/lib64/qt-3.3/bin:/usr/lib64/compat-openmpi16/bin:/usr/bin:/bin:/usr/sbin:/sbin::/sbin:/bin:/usr/sbin: NAME=autotest_config sh rpc.sh set_default_debug \"-1\" \"all -lnet -lnd -pinger\" 4 
 replay-dual test_21b: @@@@@@ FAIL: can't check if COS works: rename replied w/o COS 

In past 15 days this issue has occurred around 36 times.

Issue Links

• is duplicated by: LU-2230 replay-dual test_21b: @@@@@@ FAIL: The test cannot check whether COS works or not: all renames are replied w/o COS
• is duplicated by: LU-4104 Failure on test suite replay-dual test_21b: Not all renames are replayed. COS=1
• is related to: LU-9586 replay-dual test cases 15c 21b remove from ALWAYS_EXCEPT
• is related to: LU-4470 replay-dual test_21b: FAIL: lustre-MDT0000: ffff88007f581100 CANNOT BE SET READONLY: rc = -95

Comments

[Gerrit Updater]

Hongchao Zhang ([e-mail]) uploaded a new patch: https://review.whamcloud.com/31538
Subject: LU-8333 test: use async_commit_count to test COS
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: c78f0c429012757b1b5d0c6bd5a368557f1d929a

[Hongchao Zhang]

the patch https://review.whamcloud.com/#/c/26030/ has been updated.

[Saurabh Tandan (Inactive)]

Re-opening the ticket. Recently, replay-dual test 21b was run removing it from Always_Except list and the issue still persists.
https://testing.hpdd.intel.com/test_sets/3dc43c8a-7d49-11e7-9ce0-5254006e85c2
Looking at the comments history on this ticket it appears that Hongchao's proposed solution https://review.whamcloud.com/#/c/26030/ never got landed.

[Peter Jones]

Landed for 2.10

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/26631/
Subject: LU-8333 test: Add replay-dual 21b to ALWAYS_EXCEPT
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 83296931009c8bdc423a2374c921e610786870c3

[Gerrit Updater]

James Casper ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26631
Subject: LU-8333: Excepts replay-dual test_21b while a new COS check test can be developed
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 95fde52018764c4a88c1d5ff479063631cb9cb6b

[Gerrit Updater]

Hongchao Zhang ([e-mail]) uploaded a new patch: https://review.whamcloud.com/26030
Subject: LU-8333 test: add more conflict operations
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: dad5c6ded7336ee7dba2a1cf79833840032b7306

[Joseph Gmitter (Inactive)]

Hi Mike,

Can you comment on the validity of this test case in relation to commit on share? Is it a valid test case, providing useful testing?

Thanks.
Joe

[Gerrit Updater]

Minh Diep ([e-mail]) uploaded a new patch: https://review.whamcloud.com/25848
Subject: LU-8333 test: disable replay-dual 21b
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 9d046935201bf2ea6b0e788f16c2d1bdaf843577

[Hongchao Zhang]

I have checked the recent failures and found the transactions of the client2 (intended to introduce the transaction gap)
has been committed before failing over the MDT, which cause the client1 to recover successfully.

I'll consider some new means to test the COS feature.