[LU-4448] Test failure timeout on sanity-quota test_7c: client sysrq and reboot

• Status: Open
• Project: Lustre
• Affects Version/s: Lustre 2.6.0, Lustre 2.7.0
• Type: Bug
• Priority: Minor
• Reporter: Maloo
• Assignee: WC Triage
• Resolution: Unresolved
• Labels: performance, zfs
• Severity: 3

Description

This issue was created by maloo for Nathaniel Clark <[e-mail]>

This issue relates to the following test suite run:
https://maloo.whamcloud.com/test_sets/7d97fc1a-7696-11e3-9ce8-52540035b04c
https://maloo.whamcloud.com/test_sets/e5c9bb9c-5987-11e3-9a87-52540035b04c

The sub-test test_7c failed with the following error:

test failed to respond and timed out

Info required for matching: sanity-quota 7c

Issue Links

• is related to: LU-7737 osd_handler.c:2777:osd_object_destroy...
• is related to: LU-8326 sanity-quota test_21 test failed to r...

Comments

[Oleg Drokin]

Looks like regular zfs slowness to me?

[Nathaniel Clark]

It does look like zfs slowness.

Also happens on sanity-quota/7a (review-zfs)
https://maloo.whamcloud.com/test_sets/7854d30e-7b14-11e3-8b19-52540035b04c

[Sarah Liu]

MDS reboot in this case:
server and client: lustre-master build #2690
https://testing.hpdd.intel.com/test_sets/d28e8234-5403-11e4-9c8a-5254006e85c2

[James Nunez (Inactive)]

Looks like another occurrence on master pre-2.8:
2015-08-27 05:13:08 - https://testing.hpdd.intel.com/test_sets/ebfea220-4cbf-11e5-a5a5-5254006e85c2
2015-10-30 05:00:43 - https://testing.hpdd.intel.com/test_sets/3a516c26-7f08-11e5-a7b1-5254006e85c2
2015-11-02 12:33:04 - https://testing.hpdd.intel.com/test_sets/d9ac97aa-8192-11e5-9c69-5254006e85c2
2015-11-03 06:12:31 - https://testing.hpdd.intel.com/test_sets/0f3bc9ca-821c-11e5-aa47-5254006e85c2
2015-11-03 21:57:08 - https://testing.hpdd.intel.com/test_sets/e88f987a-829c-11e5-aa47-5254006e85c2
2015-11-11 04:17:11 - https://testing.hpdd.intel.com/test_sets/f06a7394-885a-11e5-84a2-5254006e85c2
2015-12-07 17:52:27 - https://testing.hpdd.intel.com/test_sets/e8d17c96-9d3a-11e5-8e88-5254006e85c2

[Jinshan Xiong (Inactive)]

new occurrence at: https://testing.hpdd.intel.com/test_sessions/0660da7c-cc02-11e5-901d-5254006e85c2

I have seen this happened in my recent patch twice, but I didn't see any occurrence in the previous version of my patch. I wonder if there is a recent landing worsening the problem

[Andreas Dilger]

The recent failures of test_7c are related to LU-7737, with the ASSERTION message on the MDS1 console log.

[Jinshan Xiong (Inactive)]

hmm.. I checked it again and there is no such message in the mds console log. Anyway thanks for the info and I will rebase my patch on LU-7737.

[Richard Henwood (Inactive)]

[delete unrelated test failures]

[nasf (Inactive)]

+1 on master:
https://testing.hpdd.intel.com/test_sets/65a795f8-bf49-11e6-92c6-5254006e85c2

[Steve Guminski (Inactive)]

Another on master:

https://testing.hpdd.intel.com/test_sets/cbab0a42-ee74-11e6-b2cc-5254006e85c2

[Minh Diep]

+1 on master

https://testing.hpdd.intel.com/test_sets/2e883e78-f8a7-11e6-887f-5254006e85c2