[LU-10956] sanity-pfl test_3: Kernel panic - not syncing: Pool has encountered an uncorrectable I/O failure and the failure mode property for this pool is set to panic

• Type: Bug
• Resolution: Unresolved
• Priority: Minor
• Affects Version/s: Lustre 2.12.0, Lustre 2.14.0, Lustre 2.12.5, Lustre 2.12.8, Lustre 2.15.3
• Labels: zfs
• Severity: 3
• Assignee: WC Triage
• Reporter: Maloo

Description

This issue was created by maloo for sarah_lw <[e-mail]>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/252abdaa-477b-11e8-95c0-52540065bddc

test_3 failed with the following error:

Test crashed during sanity-pfl test_3

env: RHEL7 zfs DNE tag-2.11.51

this is the trace found in kernel-crash.log

[34408.762645] Lustre: DEBUG MARKER: dmesg
[34409.519801] Lustre: DEBUG MARKER: /usr/sbin/lctl mark == sanity-pfl test 3: Delete component from existing file ============================================ 04:43:50 \(1524545030\)
[34409.734904] Lustre: DEBUG MARKER: == sanity-pfl test 3: Delete component from existing file ============================================ 04:43:50 (1524545030)
[34434.509312] Lustre: lustre-OST0006: Client lustre-MDT0001-mdtlov_UUID (at 10.9.4.25@tcp) reconnecting
[34434.512144] Lustre: lustre-OST0006: Client lustre-MDT0003-mdtlov_UUID (at 10.9.4.25@tcp) reconnecting
[34434.512149] Lustre: Skipped 7 previous similar messages
[34434.516050] WARNING: MMP writes to pool 'lustre-ost2' have not succeeded in over 20s; suspending pool
[34434.516059] Kernel panic - not syncing: Pool 'lustre-ost2' has encountered an uncorrectable I/O failure and the failure mode property for this pool is set to panic.
[34434.516071] CPU: 0 PID: 16454 Comm: mmp Tainted: P OE ------------ 3.10.0-693.21.1.el7_lustre.x86_64 #1
[34434.516072] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007
[34434.516077] Call Trace:
[34434.516133] [<ffffffff816ae7c8>] dump_stack+0x19/0x1b
[34434.516137] [<ffffffff816a8634>] panic+0xe8/0x21f
[34434.516443] [<ffffffffc05734a6>] zio_suspend+0x106/0x110 [zfs]
[34434.516470] [<ffffffffc04fa322>] mmp_thread+0x322/0x4a0 [zfs]
[34434.516491] [<ffffffffc04fa000>] ? mmp_write_done+0x1d0/0x1d0 [zfs]
[34434.516528] [<ffffffffc03aefc3>] thread_generic_wrapper+0x73/0x80 [spl]
[34434.516532] [<ffffffffc03aef50>] ? __thread_exit+0x20/0x20 [spl]
[34434.516555] [<ffffffff810b4031>] kthread+0xd1/0xe0
[34434.516558] [<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40
[34434.516574] [<ffffffff816c0577>] ret_from_fork+0x77/0xb0
[34434.516577] [<ffffffff810b3f60>] ? insert_kthread_work+0x40/0x40

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity-pfl test_3 - Test crashed during sanity-pfl test_3

Issue Links

• is duplicated by: LU-15261 sanity: 160j - storage timeout
• is duplicated by: LU-11217 conf-sanity: test_32c client crashed
• is duplicated by: LU-12281 replay-single: ZFS mmp crashed
• is duplicated by: LU-12393 insanity: Kernel panic - not syncing: Pool 'lustre-ost6' has encountered an uncorrectable I/O failure and the failure mode property for this pool is set to panic.
• is related to: LU-13242 sanity-pfl: WARNING: MMP writes to pool have not succeeded in over 60s; suspending pool
• is related to: LU-9845 ost-pools test_22 hangs with ‘WARNING: Pool 'lustre-mdt1' has encountered an uncorrectable I/O failure and has been suspended.’