[LU-14824] sanity test_413a: timeout

• Type: Bug
• Resolution: Fixed
• Priority: Minor
• Fix Version/s: Lustre 2.16.0, Lustre 2.15.5
• Severity: 3
• Assignee: Lai Siyao
• Reporter: Maloo

Description

This issue was created by maloo for S Buisson <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/a8d333cd-1b69-4f00-9829-2590702b0c0e

test_413a failed with the following error:

Timeout occurred after 492 mins, last suite running was sanity

The test is blocked for an unknown reason, as nothing is visible in the console of the client or server nodes. Last message in test log is:

Mkdir (stripe_count 3) roundrobin:

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity test_413a - Timeout occurred after 492 mins, last suite running was sanity

Issue Links

• is related to: LU-16507 sanity test_413a: division by 0 (error token is "min")
• is related to: LU-14659 sanity test_413a: subdirs shouldn't be evenly distributed

Comments

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49799
Subject: LU-14824 test: sanity 413a/b unlink timeout v2
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 09dd0a091defa34ed7de2402b64f0e38dbefd9a6

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/49646/
Subject: LU-14824 Revert "test: sanity 413a/b unlink timeout"
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 601ed56575a304c15ccb6d98a252162e64ef95e9

[Andreas Dilger]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/c/fs/lustre-release/+/49646
Subject: LU-14824 Revert "test: sanity 413a/b unlink timeout"
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 46277a6a1affb8b21abb28941ff4b471d2b3bd32

[Andreas Dilger]

Patch is being reverted due to many timeouts in ldiskfs since landing.

The patch 45955 was pushed and tested on 2022-03-06, but it looks like another patch may have landed in this same code in between and caused the failure.

[Peter Jones]

Landed for 2.16

[Gerrit Updater]

"Oleg Drokin <[e-mail]>" merged in patch https://review.whamcloud.com/c/fs/lustre-release/+/45955/
Subject: LU-14824 test: sanity 413a/b unlink timeout
Project: fs/lustre-release
Branch: master
Current Patch Set:
Commit: 5ff3e400f1a74ea49b7eb9cf19715f0fae08c3f5

[Nikitas Angelinas]

+1 on master: https://testing.whamcloud.com/test_sets/46d8f39d-da3a-4039-bd02-d7c90196d7f9

[Andreas Dilger]

It is worthwhile to note that patch https://review.whamcloud.com/46734 "LU-15528 mdt: enqueue newly created object locks in TXN mode" and the later patch https://review.whamcloud.com/46733 "LU-15526 mdt: enable remote PDO lock" are about 10x faster (~150-170s vs. ~1100-3000s) when running sanity test_413a compared to unpatched systems:

https://testing.whamcloud.com/search?server_file_system_type_id=00437f32-318d-11e1-9c6d-5254004bbbd3&test_set_script_id=f9516376-32bc-11e0-aaee-52540025f9ae&sub_test_script_id=44d5fa14-70d0-11e9-a6f2-52540065bddc&start_date=2022-03-07&end_date=2022-03-09&source=sub_tests#redirect

[Gerrit Updater]

"Andreas Dilger <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/46774
Subject: LU-14824 tests: reduce sanity test_413 ZFS test time
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 502cc9bb0ee94537a77e56f3888c25f11f8790a0

[Andreas Dilger]

+1 on master: https://testing.whamcloud.com/test_sets/3720ae52-c898-40bd-9bb0-f41ea075568c

Currently failing about 2.5% of runs, but 7.5% of ZFS runs.

[Gerrit Updater]

"Lai Siyao <[e-mail]>" uploaded a new patch: https://review.whamcloud.com/45955
Subject: LU-14824 test: collect debug logs on zfs system
Project: fs/lustre-release
Branch: master
Current Patch Set: 1
Commit: 1d583433b1a4ad23a99ecb85fe4dc6858edaef20