[LU-8445] soft lockup in JBD2 on the OST

Duplicate — Wed, 5 Aug 2020 21:06:04 +0000

This issue was created by maloo for nasf <fan.yong@intel.com>

Please provide additional information about the failure here.

This issue relates to the following test suite run: https://testing.hpdd.intel.com/test_sets/a78870c0-53c2-11e6-a39e-5254006e85c2.

The sanity-hsm failed with the following stack trace on the OST:

19:25:47:[16928.052003] BUG: soft lockup - CPU#0 stuck for 22s! [jbd2/dm-4-8:31237]^M
19:25:47:[16928.052003] Modules linked in: osp(OE) ofd(OE) lfsck(OE) ost(OE) mgc(OE) osd_ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ksocklnd(OE) ptlrpc(OE) obdcl
ass(OE) lnet(OE) sha512_generic crypto_null libcfs(OE) ldiskfs(OE) dm_mod rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache xprtrdma ib_isert iscsi_target_mod i
b_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod crc_t10dif crct10dif_generic crct10dif_common ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_u
cm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ppdev parport_pc pcspkr i2c_piix4 virtio_balloon parport nfsd nfs_acl lockd auth
_rpcgss grace sunrpc ip_tables ext4 mbcache jbd2 ata_generic pata_acpi virtio_blk 8139too cirrus syscopyarea sysfillrect sysimgblt drm_kms_helper ttm serio_r
aw virtio_pci virtio_ring virtio drm ata_piix 8139cp mii i2c_core libata floppy^M19:25:47:[16928.052003] CPU: 0 PID: 31237 Comm: jbd2/dm-4-8 Tainted: G           OE  ------------   3.10.0-327.22.2.el7_lustre.x86_64 #1^M
19:25:47:[16928.052003] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2007^M
19:25:47:[16928.052003] task: ffff88004f25b980 ti: ffff8800776f8000 task.ti: ffff8800776f8000^M
19:25:47:[16928.052003] RIP: 0010:[<ffffffff8163dcd2>]  [<ffffffff8163dcd2>] _raw_spin_lock+0x32/0x50^M
19:25:47:[16928.052003] RSP: 0018:ffff8800776fbca0  EFLAGS: 00000287^M
19:25:47:[16928.052003] RAX: 0000000000002cab RBX: ffff8800776fbc90 RCX: 0000000000000c00^M
19:25:47:[16928.052003] RDX: 0000000000000b4e RSI: 0000000000000b4e RDI: ffff88003b0ed3a0^M
19:25:47:[16928.052003] RBP: ffff8800776fbca0 R08: 0000000000000202 R09: 0000000000000035^M
19:25:47:[16928.052003] R10: 0000000000000000 R11: 0000000000000001 R12: ffffffff81211940^M
19:25:47:[16928.052003] R13: 0000000000000002 R14: ffff880079969dd0 R15: 0000000000000002^M
19:25:47:[16928.052003] FS:  0000000000000000(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000^M
19:25:47:[16928.052003] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b^M
19:25:47:[16928.052003] CR2: 00007f3e50c6f4a9 CR3: 000000003f117000 CR4: 00000000000006f0^M
19:25:47:[16928.052003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000^M
19:25:47:[16928.052003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400^M
19:25:47:[16928.052003] Stack:^M
19:25:47:[16928.052003]  ffff8800776fbe40 ffffffffa016cf70 00000f5f8b4cb4da ffff8800776fbfd8^M
19:25:47:[16928.052003]  0000000800000018 ffff88004b51ff9c ffff88004b51ff00 ffff8800776fbfd8^M
19:25:47:[16928.052003]  ffff88005cd49024 ffff8800363f6000 0000000000000fdc ffff8800ffffffff^M
19:25:47:[16928.052003] Call Trace:^M
19:25:47:[16928.052003]  [<ffffffffa016cf70>] jbd2_journal_commit_transaction+0x10a0/0x19a0 [jbd2]^M
19:25:47:[16928.052003]  [<ffffffff81013588>] ? __switch_to+0xf8/0x4b0^M
19:25:47:[16928.052003]  [<ffffffffa0171d79>] kjournald2+0xc9/0x260 [jbd2]^M
19:25:47:[16928.052003]  [<ffffffff810a6ae0>] ? wake_up_atomic_t+0x30/0x30^M
19:25:47:[16928.052003]  [<ffffffffa0171cb0>] ? commit_timeout+0x10/0x10 [jbd2]^M
19:25:47:[16928.052003]  [<ffffffff810a5aef>] kthread+0xcf/0xe0^M
19:25:47:[16928.052003]  [<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140^M
19:25:47:[16928.052003]  [<ffffffff816469d8>] ret_from_fork+0x58/0x90^M
19:25:47:[16928.052003]  [<ffffffff810a5a20>] ? kthread_create_on_node+0x140/0x140^M
19:25:47:[16928.052003] Code: 89 e5 b8 00 00 02 00 f0 0f c1 07 89 c2 c1 ea 10 66 39 c2 75 02 5d c3 83 e2 fe 0f b7 f2 b8 00 80 00 00 eb 0c 0f 1f 44 00 00 f3 90 <83> e8 01 74 0a 0f b7 0f 66 39 ca 75 f1 5d c3 0f 1f 80 00 00 00 ^M

Whamcloud Community JIRA

[LU-8445] soft lockup in JBD2 on the OST