[LU-12470] sanityn test_47b: create isn't blocked

• Type: Bug
• Resolution: Fixed
• Priority: Major
• Fix Version/s: Lustre 2.14.0
• Affects Version/s: Lustre 2.13.0, Lustre 2.14.0
• Severity: 3
• Assignee: Andreas Dilger
• Reporter: Maloo

Description

This issue was created by maloo for Chris Horn <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/a0a32158-9756-11e9-8262-52540065bddc

test_47b failed with the following error:

create isn't blocked

== sanityn test 47b: pdirops: remote mkdir vs create ================================================= 10:59:51 (1561460391)
CMD: trevis-38vm9 lctl set_param fail_loc=0x80000145
fail_loc=0x80000145
CMD: trevis-38vm9 lctl set_param fail_loc=0
fail_loc=0
No conflict
lfs mkdir: dirstripe error on '/mnt/lustre/f47b.sanityn': stripe already set
lfs setdirstripe: cannot create dir '/mnt/lustre/f47b.sanityn': File exists
 sanityn test_47b: @@@@@@ FAIL: create isn't blocked 

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanityn test_41a - mkdir isn't blocked
sanityn test_41b - create isn't blocked
sanityn test_41c - link isn't blocked
sanityn test_41d - unlink isn't blocked
sanityn test_41e - rename isn't blocked
sanityn test_41f - rename isn't blocked
sanityn test_41g - getattr isn't blocked
sanityn test_42a - mkdir isn't blocked
sanityn test_42b - create isn't blocked
sanityn test_42c - link isn't blocked
sanityn test_42d - unlink isn't blocked
sanityn test_42e - rename isn't blocked
sanityn test_42f - rename isn't blocked
sanityn test_42g - getattr isn't blocked
sanityn test_43a - mkdir isn't blocked
sanityn test_43b - create isn't blocked
sanityn test_43c - link isn't blocked
sanityn test_43d - unlink isn't blocked
sanityn test_43e - rename isn't blocked
sanityn test_43f - rename isn't blocked
sanityn test_43g - getattr isn't blocked
sanityn test_43i - remote mkdir isn't blocked
sanityn test_44a - mkdir isn't blocked
sanityn test_44b - create isn't blocked
sanityn test_44c - link isn't blocked
sanityn test_44d - unlink isn't blocked
sanityn test_44e - rename isn't blocked
sanityn test_44f - rename isn't blocked
sanityn test_44g - getattr isn't blocked
sanityn test_44i - remote mkdir isn't blocked
sanityn test_45a - mkdir isn't blocked
sanityn test_45b - create isn't blocked
sanityn test_45c - link isn't blocked
sanityn test_45d - unlink isn't blocked
sanityn test_45e - rename isn't blocked
sanityn test_45f - rename isn't blocked
sanityn test_45g - getattr isn't blocked
sanityn test_46a - mkdir isn't blocked
sanityn test_46b - create isn't blocked
sanityn test_46c - link isn't blocked
sanityn test_46d - unlink isn't blocked
sanityn test_46e - rename isn't blocked
sanityn test_46f - rename isn't blocked
sanityn test_46g - getattr isn't blocked
sanityn test_46i - remote mkdir isn't blocked
sanityn test_47a - mkdir isn't blocked
sanityn test_47b - create isn't blocked
sanityn test_47c - link isn't blocked
sanityn test_47d - unlink isn't blocked
sanityn test_47e - rename isn't blocked
sanityn test_47f - rename isn't blocked
sanityn test_47g - getattr isn't blocked

Issue Links

• duplicates: LU-12407 sanityn test_42d: unlink isn't blocked
• is duplicated by: LU-10870 sanityn test 40a, 40b, 40c, 40d, 40e fail with 'create is blocked'
• is duplicated by: LU-14746 Interop: sanityn test_41b fails with 'create isn't blocked'
• is duplicated by: LU-4575 Test failure sanityn test_41g: getattr isn't blocked
• is duplicated by: LU-6641 sanityn test_46i- FAIL: remote mkdir isn't blocked
• is duplicated by: LU-8759 sanityn test_41c: link isn't blocked
• is duplicated by: LU-10874 sanityn test_40b: @@@@@@ FAIL: create is blocked
• is duplicated by: LU-12408 sanityn test_44g: getattr isn't blocked
• is duplicated by: LU-12435 sanityn: test_44i: @@@@@@ FAIL: remote mkdir isn't blocked
• is duplicated by: LU-12437 sanityn test_44f: rename isn't blocked
• is duplicated by: LU-12449 sanityn test_47b: create must fail
• is duplicated by: LU-12466 sanityn test_45a fails with: mkdir isn't blocked
• is duplicated by: LU-12551 sanityn test_46e: rename isn't blocked
• is duplicated by: LU-12576 sanityn: test_43d 'unlink isn't blocked'
• is duplicated by: LU-13097 sanityn test_47b: create must fail
• is related to: LU-10754 sanityn test 47b fails with 'create must fail'
• is related to: LU-12689 sanityn test_42b: create isn't blocked

Comments

[James Nunez (Inactive)]

I'm going to open a new ticket for this, but thought I'd leave a note here. We are still seeing this issue for interop testing between master clients and 2.12.6 servers.

One such failure is at https://testing.whamcloud.com/test_sets/d3a83e0b-15be-4799-b8ba-c264835472e2 .

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/37304/
Subject: LU-12470 tests: increase pdirops timeout
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: b35f50c96c608ba650a5b3cf29fa129e01025549

[Gerrit Updater]

Andreas Dilger ([e-mail]) uploaded a new patch: https://review.whamcloud.com/37304
Subject: LU-12470 tests: increase pdirops timeout
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: a42f438a4acac3145cd5b7b178a737dbacc43d32

[Arshad Hussain]

Seen on master: https://testing.whamcloud.com/sub_tests/863eeb94-2b1c-11ea-b0f4-52540065bddc

[Emoly Liu]

+1 on master: https://testing.whamcloud.com/test_sets/bca0aec6-0abe-11ea-b934-52540065bddc

[Chris Horn]

+1 on master https://testing.whamcloud.com/test_sessions/7a906b8b-f8ee-48b5-91bc-d3628a032560

[Chris Horn]

+1 on master https://testing.whamcloud.com/test_sessions/3f181300-d3f8-465a-88de-95756bf58f3c

[Emoly Liu]

+1 on master with the patch: https://testing.whamcloud.com/test_sets/60b8ee10-e03a-11e9-a197-52540065bddc

[Gerrit Updater]

Oleg Drokin ([e-mail]) merged in patch https://review.whamcloud.com/35865/
Subject: LU-12470 tests: clear MDT-MDT locks for pdo tests
Project: fs/lustre-release
Branch: b2_12
Current Patch Set:
Commit: cf3506ccb25f4a4f57150e51e7a8bde4cee80a26

[Andreas Dilger]

It seems that there are now two different failure modes for these tests:

• occasional review test failures for patches similar to what happened before the LU-12470 patch landed, so it isn't clear whether this patch really fixed anything?
• complete failure of subtests in the 40-46 range when running tests with 2.12.2 MDS/OSS and reporting locks not being canceled by lru_size=clear:

  == sanityn test 41b: pdirops: create vs create ======================================================= 15:25:02 (1565882702)
  CMD: trevis-20vm12 /usr/sbin/lctl set_param -n ldlm.namespaces.*mdt*.lru_size=clear
  CMD: trevis-20vm12 /usr/sbin/lctl get_param ldlm.namespaces.*mdt*.lock_unused_count ldlm.namespaces.*mdt*.lock_count
  ldlm.namespaces.mdt-lustre-MDT0000_UUID.lock_count=43
  

The testing interop problem may relate to a patch in 2.13 on how the LRU is cleared (always waiting for locks to be cancelled?), but I can't find the patch in question.

I think it makes sense to reopen this patch and not mark it resolved for 2.13, and either revert the patch (since it breaks testing interop), or figure out what is causing the testing interop issues and include the fix to 2.12.3 (though this is sub-optimal for some reasons).

[Chris Horn]

Opened https://jira.whamcloud.com/browse/LU-12689 to track new failure