[LU-16914] sanity-scrub test_8: sanity-scrub test_8 crashed

Tue, 20 Jun 2023 00:37:10 +0000

This issue was created by maloo for Minh Diep <mdiep@whamcloud.com>

This issue relates to the following test suite run: https://testing.whamcloud.com/test_sets/f281d578-4dfd-48ef-b75f-c7ad0e309b34

test_8 failed with the following error:

onyx-117vm4 crashed during sanity-scrub test_8

Test session details:
clients: https://build.whamcloud.com/job/lustre-master/4435 - 5.14.0-284.11.1.el9_2.x86_64
servers: https://build.whamcloud.com/job/lustre-master/4435 - 5.14.0-284.11.1_lustre.el9.x86_64

<<Please provide additional information about the failure here>>

[ 1550.890567] Lustre: DEBUG MARKER: /usr/sbin/lctl lfsck_stop -M lustre-MDT0000
[ 1575.440253] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [lfsck:92306]
[ 1575.448514] Modules linked in: dm_flakey osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) lustre(OE) lmv(OE) mdc(OE) lov(OE) osc(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) ldiskfs(OE) libcfs(OE) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs rfkill sunrpc intel_rapl_msr intel_rapl_common virtio_balloon pcspkr i2c_piix4 joydev drm fuse dm_mod ext4 mbcache jbd2 ata_generic crct10dif_pclmul crc32_pclmul ata_piix crc32c_intel libata virtio_net ghash_clmulni_intel net_failover failover virtio_blk serio_raw [last unloaded: dm_flakey]
[ 1575.454637] CPU: 1 PID: 92306 Comm: lfsck Kdump: loaded Tainted: G OE -------- — 5.14.0-284.11.1_lustre.el9.x86_64 #1
[ 1575.456052] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 1575.456790] RIP: 0010:kvm_wait+0x4b/0x50
[ 1575.457357] Code: 74 1d fb c3 cc cc cc cc 0f b6 07 40 38 c6 75 f3 eb 07 0f 00 2d 02 c5 fa 00 f4 c3 cc cc cc cc eb 07 0f 00 2d f3 c4 fa 00 fb f4 <c3> cc cc cc cc 0f 1f 44 00 00 8b 05 f5 8a 9c 01 83 f8 ff 74 0a 85
[ 1575.459478] RSP: 0018:ffffacd085ea7e08 EFLAGS: 00000246
[ 1575.460138] RAX: 0000000000000003 RBX: ffff95347fd2bac0 RCX: 0000000000000008
[ 1575.461003] RDX: 0000000000000000 RSI: 0000000000000003 RDI: ffff9533ef635020
[ 1575.461868] RBP: ffff9533ef635020 R08: ffff95347ffc2540 R09: 0000000000000000
[ 1575.462735] R10: ffff95347fd2bac0 R11: 0000000000000001 R12: 0000000000000000
[ 1575.463611] R13: 0000000000000001 R14: 0000000000000100 R15: 0000000000080000
[ 1575.464474] FS: 0000000000000000(0000) GS:ffff95347fd00000(0000) knlGS:0000000000000000
[ 1575.465445] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1575.466161] CR2: 00007fc9a0c7d0a0 CR3: 00000000828c4004 CR4: 00000000001706e0
[ 1575.467038] Call Trace:
[ 1575.467409] <TASK>
[ 1575.467749] __pv_queued_spin_lock_slowpath+0x26f/0x2b0
[ 1575.468450] _raw_spin_lock+0x25/0x30
[ 1575.468971] lfsck_pos_fill+0x71/0x120 [lfsck]
[ 1575.469693] lfsck_master_engine+0x1aa/0xb20 [lfsck]
[ 1575.470349] ? try_to_wake_up+0x1e5/0x530
[ 1575.470892] ? cpuacct_percpu_seq_show+0x10/0x10
[ 1575.471484] ? lfsck_post+0x220/0x220 [lfsck]
[ 1575.472077] kthread+0xd9/0x100
[ 1575.472530] ? kthread_complete_and_exit+0x20/0x20
[ 1575.473142] ret_from_fork+0x22/0x30
[ 1575.473645] </TASK>
[ 1575.473977] Kernel panic - not syncing: softlockup: hung tasks
[ 1575.474706] CPU: 1 PID: 92306 Comm: lfsck Kdump: loaded Tainted: G OEL -------- — 5.14.0-284.11.1_lustre.el9.x86_64 #1
[ 1575.476105] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 1575.476821] Call Trace:
[ 1575.477191] <IRQ>

VVVVVVV DO NOT REMOVE LINES BELOW, Added by Maloo for auto-association VVVVVVV
sanity-scrub test_8 - onyx-117vm4 crashed during sanity-scrub test_8

Whamcloud Community JIRA

[LU-16914] sanity-scrub test_8: sanity-scrub test_8 crashed