[LU-10075] sanity test_123a: kernel panic on client

Wed, 5 Aug 2020 21:17:10 +0000

This issue was created by maloo for Bob Glossman <bob.glossman@intel.com>

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/3df62e80-a930-11e7-b78a-5254006e85c2.

The sub-test test_123a failed with the following error:

Timeout occurred after 156 mins, last suite running was sanity, restarting cluster to continue tests

console log of client shows:

[ 5822.968994] Lustre: DEBUG MARKER: == sanity test 123a: verify statahead work =========================================================== 09:00:10 (1507107610)
[ 5823.458464] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls 100 files without statahead: 0 sec
[ 5823.626713] Lustre: DEBUG MARKER: ls 100 files without statahead: 0 sec
[ 5823.929957] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls 100 files with statahead: 0 sec
[ 5824.103279] Lustre: DEBUG MARKER: ls 100 files with statahead: 0 sec
[ 5826.482911] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls 1000 files without statahead: 1 sec
[ 5826.653705] Lustre: DEBUG MARKER: ls 1000 files without statahead: 1 sec
[ 5827.357208] Lustre: DEBUG MARKER: /usr/sbin/lctl mark ls 1000 files with statahead: 0 sec
[ 5827.522515] Lustre: DEBUG MARKER: ls 1000 files with statahead: 0 sec
[ 5829.444255] general protection fault: 0000 [#1] SMP 
[ 5829.444887] Modules linked in: loop lustre(OE) obdecho(OE) mgc(OE) lov(OE) osc(OE) mdc(OE) lmv(OE) fid(OE) fld(OE) ptlrpc_gss(OE) ptlrpc(OE) obdclass(OE) ksocklnd(OE) lnet(OE) libcfs(OE) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod crc_t10dif crct10dif_generic ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_core iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul nfsd glue_helper ablk_helper cryptd parport_pc i2c_piix4 i2c_core joydev pcspkr parport virtio_balloon nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi ata_piix libata virtio_blk 8139too crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel virtio_ring serio_raw virtio floppy 8139cp mii
[ 5829.453891] CPU: 1 PID: 2961 Comm: createmany Tainted: G           OE  ------------   3.10.0-693.1.1.el7.x86_64 #1
[ 5829.454878] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[ 5829.455434] task: ffff88006a643f40 ti: ffff88001d9a8000 task.ti: ffff88001d9a8000
[ 5829.456156] RIP: 0010:[<ffffffff8132bc5c>]  [<ffffffff8132bc5c>] memcmp+0xc/0x50
[ 5829.456925] RSP: 0018:ffff88001d9abb30  EFLAGS: 00010202
[ 5829.457441] RAX: 0000000000000009 RBX: 0000000000000000 RCX: 0000000000000000
[ 5829.458124] RDX: 0000000000000020 RSI: ffff000000000048 RDI: ffff88001d9abd30
[ 5829.458813] RBP: ffff88001d9abb30 R08: 0000000000000003 R09: 00000000c000094a
[ 5829.459521] R10: 000000000000000a R11: 00000002c0000409 R12: ffff88001d9abd30
[ 5829.460197] R13: 0000000000000003 R14: ffff000000000000 R15: ffff880044d8e900
[ 5829.460883] FS:  00007f6d37786740(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
[ 5829.461663] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 5829.462216] CR2: 0000000001d11890 CR3: 0000000046f19000 CR4: 00000000000406e0
[ 5829.462915] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 5829.463607] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 5829.464286] Stack:
[ 5829.464499]  ffff88001d9abb40 ffffffffc090a9e7 ffff88001d9abb90 ffffffffc05cfe07
[ 5829.465288]  000000003698b300 0000000000000000 ffff88001d9abbc0 ffff88005c97ec00
[ 5829.466088]  ffff88001d9abd30 ffff880044d8e900 000000000000000d 0000000000000000
[ 5829.466890] Call Trace:
[ 5829.467239]  [<ffffffffc090a9e7>] ldlm_res_hop_keycmp+0x17/0x30 [ptlrpc]
[ 5829.467965]  [<ffffffffc05cfe07>] cfs_hash_bd_lookup_intent+0x57/0x160 [libcfs]
[ 5829.468685]  [<ffffffffc05cff26>] cfs_hash_bd_lookup_locked+0x16/0x20 [libcfs]
[ 5829.469408]  [<ffffffffc090df66>] ldlm_resource_get+0x86/0xa30 [ptlrpc]
[ 5829.470073]  [<ffffffffc0905dce>] ldlm_lock_match+0xae/0xa70 [ptlrpc]
[ 5829.470703]  [<ffffffffc05bf395>] ? cfs_trace_unlock_tcd+0x35/0x90 [libcfs]
[ 5829.471543]  [<ffffffffc0a5b1f2>] mdc_lock_match+0xb2/0x180 [mdc]
[ 5829.472191]  [<ffffffffc08c43d9>] lmv_lock_match+0x239/0x5f0 [lmv]
[ 5829.472886]  [<ffffffffc07422b9>] ? lprocfs_counter_add+0xf9/0x160 [obdclass]
[ 5829.473637]  [<ffffffffc0bd19b6>] ll_file_release+0x626/0xa70 [lustre]
[ 5829.474279]  [<ffffffff81202fb9>] __fput+0xe9/0x260
[ 5829.474759]  [<ffffffff8120321e>] ____fput+0xe/0x10
[ 5829.475247]  [<ffffffff810ad247>] task_work_run+0xa7/0xf0
[ 5829.475782]  [<ffffffff8102ab62>] do_notify_resume+0x92/0xb0
[ 5829.476341]  [<ffffffff816b527d>] int_signal+0x12/0x17
[ 5829.476845] Code: 73 09 48 8b 0c cf 48 85 c9 75 cc b8 ea ff ff ff 0f 1f 80 00 00 00 00 5d c3 66 0f 1f 44 00 00 55 48 85 d2 48 89 e5 74 3c 0f b6 07 <0f> b6 0e 29 c8 75 27 48 83 ea 01 31 c9 eb 1a 0f 1f 44 00 00 44 
[ 5829.479955] RIP  [<ffffffff8132bc5c>] memcmp+0xc/0x50
[ 5829.480470]  RSP <ffff88001d9abb30>

Info required for matching: sanity 123a

Whamcloud Community JIRA

[LU-10075] sanity test_123a: kernel panic on client