[ 147.203822] Lustre: soaked-MDT0000: Connection restored to 880972f7-970d-b37a-1a61-2887fd2e2d91 (at 192.168.1.142@o2ib100)
[ 147.203825] Lustre: Skipped 6 previous similar messages
[ 156.268838] Lustre: soaked-MDT0000: Connection restored to 192.168.1.110@o2ib10 (at 192.168.1.110@o2ib10)
[ 156.279660] Lustre: Skipped 24 previous similar messages
[ 173.233671] Lustre: MGS: Connection restored to 9665679f-f79b-1bf9-6fd6-abc25509729a (at 192.168.1.131@o2ib100)
[ 173.245048] Lustre: Skipped 28 previous similar messages
[ 190.236882] Lustre: soaked-MDT0000: Recovery over after 0:43, of 32 clients 32 recovered and 0 were evicted.
[ 218.357311] Lustre: MGS: Connection restored to 192.168.1.105@o2ib10 (at 192.168.1.105@o2ib10)
[ 218.368494] Lustre: Skipped 12 previous similar messages
[ 806.665594] Lustre: soaked-MDT0000: Client 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100) reconnecting
[ 806.678906] Lustre: soaked-MDT0000: Connection restored to 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100)
[ 806.692353] Lustre: Skipped 9 previous similar messages
[ 876.738766] Lustre: soaked-MDT0000: Client 60b5ead5-442a-039e-5d32-f9759a392209 (at 192.168.1.128@o2ib100) reconnecting
[ 1020.276376] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[ 1020.291053] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[ 1020.305862] Lustre: Skipped 1 previous similar message
[ 1216.697961] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[ 1239.749007] Lustre: soaked-MDT0000: Client 31859d53-53e2-75a2-da2d-75f6bdca7375 (at 192.168.1.118@o2ib100) reconnecting
[ 1300.750770] Lustre: soaked-MDT0000: Client 6f6036ce-2e3a-7e95-1c07-11e5d5a627fe (at 192.168.1.124@o2ib100) reconnecting
[ 1300.765260] Lustre: soaked-MDT0000: Connection restored to 5a0e6555-4c2d-39c9-90c4-a332f62c8353 (at 192.168.1.124@o2ib100)
[ 1300.779749] Lustre: Skipped 2 previous similar messages
[ 1385.754795] Lustre: soaked-MDT0000: Client d8128f9f-f4c7-b311-df64-40bd3b032886 (at 192.168.1.127@o2ib100) reconnecting
[ 1469.707717] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[ 1542.764014] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[ 1742.697552] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[ 1742.712158] Lustre: Skipped 2 previous similar messages
[ 1818.700200] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[ 1818.714597] Lustre: Skipped 7 previous similar messages
[ 1960.700811] Lustre: 3698:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1494355306/real 1494355306] req@ffff8808053c2d00 x1566943337184128/t0(0) o104->soaked-MDT0000@192.168.1.123@o2ib100:15/16 lens 296/224 e 0 to 1 dl 1494355313 ref 1 fl Rpc:X/0/ffffffff rc 0/-1
[ 2035.771561] Lustre: soaked-MDT0000: Client 54aab9cb-1b41-13ac-e1b9-f63d1e78e154 (at 192.168.1.137@o2ib100) reconnecting
[ 2035.785133] Lustre: Skipped 4 previous similar messages
[ 2540.719596] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[ 2540.735126] Lustre: Skipped 5 previous similar messages
[ 2588.807156] Lustre: soaked-MDT0000: Client e81e92b7-aede-6d47-4552-8475e879b939 (at 192.168.1.134@o2ib100) reconnecting
[ 2588.822296] Lustre: Skipped 3 previous similar messages
[ 3168.746656] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[ 3168.761959] Lustre: Skipped 8 previous similar messages
[ 3271.274472] Lustre: soaked-MDT0000: Client a60139c3-b36a-37dd-9360-e306d02df1da (at 192.168.1.131@o2ib100) reconnecting
[ 3271.289131] Lustre: Skipped 8 previous similar messages
[ 4071.250600] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[ 4071.265568] Lustre: Skipped 3 previous similar messages
[ 4071.274511] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[ 4071.289924] Lustre: Skipped 4 previous similar messages
[ 4107.234610] Lustre: 3702:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1494357452/real 1494357452] req@ffff8807c13c2a00 x1566943434075904/t0(0) o104->soaked-MDT0000@192.168.1.133@o2ib100:15/16 lens 296/224 e 0 to 1 dl 1494357459 ref 1 fl Rpc:X/0/ffffffff rc 0/-1
[ 4203.237982] Lustre: 3690:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1494357548/real 1494357548] req@ffff8807e682d700 x1566943439034896/t0(0) o104->soaked-MDT0000@192.168.1.138@o2ib100:15/16 lens 296/224 e 0 to 1 dl 1494357555 ref 1 fl Rpc:X/0/ffffffff rc 0/-1
[ 4731.411273] Lustre: soaked-MDT0000: Client dc7bce1e-8cc2-256d-8a24-aa9176826892 (at 192.168.1.125@o2ib100) reconnecting
[ 4731.426572] Lustre: Skipped 2 previous similar messages
[ 4731.435634] Lustre: soaked-MDT0000: Connection restored to eee4fa14-7423-c90d-6b47-20743657a666 (at 192.168.1.125@o2ib100)
[ 4731.451200] Lustre: Skipped 2 previous similar messages
[ 4852.341464] Lustre: MGS: Received new LWP connection from 192.168.1.117@o2ib100, removing former export from same NID
[ 5679.798406] Lustre: MGS: Received new LWP connection from 192.168.1.125@o2ib100, removing former export from same NID
[ 5679.813587] Lustre: MGS: Connection restored to eee4fa14-7423-c90d-6b47-20743657a666 (at 192.168.1.125@o2ib100)
[ 5679.827819] Lustre: Skipped 5 previous similar messages
[ 5815.372459] Lustre: soaked-MDT0000: Client f75498d6-bc9c-d0bf-fdad-54dda94a52b2 (at 192.168.1.123@o2ib100) reconnecting
[ 5815.387540] Lustre: Skipped 4 previous similar messages
[ 6707.068214] Lustre: 3465:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360052/real 0] req@ffff88009c88a700 x1566943440356288/t0(0) o13->soaked-OST0009-osc-MDT0000@192.168.1.105@o2ib10:7/4 lens 224/368 e 0 to 1 dl 1494360059 ref 2 fl Rpc:X/0/ffffffff rc 0/-1
[ 6707.109009] Lustre: soaked-OST0009-osc-MDT0000: Connection to soaked-OST0009 (at 192.168.1.105@o2ib10) was lost; in progress operations using this service will wait for recovery to complete
[ 6708.204232] Lustre: 3468:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360053/real 0] req@ffff8803f36fda00 x1566943440356384/t0(0) o13->soaked-OST0003-osc-MDT0000@192.168.1.105@o2ib10:7/4 lens 224/368 e 0 to 1 dl 1494360060 ref 2 fl Rpc:X/0/ffffffff rc 0/-1
[ 6708.245384] Lustre: soaked-OST0003-osc-MDT0000: Connection to soaked-OST0003 (at 192.168.1.105@o2ib10) was lost; in progress operations using this service will wait for recovery to complete
[ 6709.700307] Lustre: 3459:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360055/real 0] req@ffff8803d9a4b300 x1566943440356528/t0(0) o13->soaked-OST000f-osc-MDT0000@192.168.1.105@o2ib10:7/4 lens 224/368 e 0 to 1 dl 1494360062 ref 2 fl Rpc:X/0/ffffffff rc 0/-1
[ 6709.742155] Lustre: 3459:0:(client.c:2114:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 6709.756469] Lustre: soaked-OST000f-osc-MDT0000: Connection to soaked-OST000f (at 192.168.1.105@o2ib10) was lost; in progress operations using this service will wait for recovery to complete
[ 6709.781932] Lustre: Skipped 1 previous similar message
[ 6712.790413] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360059/real 0] req@ffff8803f36f8000 x1566943440356832/t0(0) o8->soaked-OST0009-osc-MDT0000@192.168.1.105@o2ib10:28/4 lens 520/544 e 0 to 1 dl 1494360065 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
[ 6770.792424] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360112/real 0] req@ffff880360faec00 x1566943440361664/t0(0) o8->soaked-OST000f-osc-MDT0000@192.168.1.105@o2ib10:28/4 lens 520/544 e 0 to 1 dl 1494360123 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
[ 6770.834398] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) Skipped 3 previous similar messages
[ 6792.429530] Lustre: soaked-MDT0000: Client 6985bb7d-8a06-d23c-3360-00e6c4a7e369 (at 192.168.1.143@o2ib100) reconnecting
[ 6792.445246] Lustre: Skipped 1 previous similar message
[ 6792.454636] Lustre: soaked-MDT0000: Connection restored to 7ea3a484-c027-1559-8163-dc85944c882c (at 192.168.1.143@o2ib100)
[ 6792.470642] Lustre: Skipped 2 previous similar messages
[ 6797.703361] LNet: 3447:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.105@o2ib10: 2 seconds
[ 6850.795225] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360187/real 0] req@ffff880360faef00 x1566943440366816/t0(0) o8->soaked-OST0015-osc-MDT0000@192.168.1.105@o2ib10:28/4 lens 520/544 e 0 to 1 dl 1494360203 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
[ 6850.837297] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) Skipped 3 previous similar messages
[ 6873.706004] LNet: 3447:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Timed out tx for 192.168.1.105@o2ib10: 3 seconds
[ 6873.721194] LNet: 3447:0:(o2iblnd_cb.c:3198:kiblnd_check_conns()) Skipped 1 previous similar message
[ 6875.796087] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360212/real 0] req@ffff880360faa700 x1566943440368512/t0(0) o8->soaked-OST0003-osc-MDT0000@192.168.1.105@o2ib10:28/4 lens 520/544 e 0 to 1 dl 1494360228 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
[ 6875.837956] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 6924.482528] Lustre: MGS: haven't heard from client 6d59f9e1-ce44-7010-80ac-34a863a225a8 (at 192.168.1.105@o2ib10) in 230 seconds. I think it's dead, and I am evicting it. exp ffff8803ef392c00, cur 1494360276 expire 1494360126 last 1494360046
[ 6930.797983] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360262/real 0] req@ffff880360fabf00 x1566943440371952/t0(0) o8->soaked-OST000f-osc-MDT0000@192.168.1.105@o2ib10:28/4 lens 520/544 e 0 to 1 dl 1494360283 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
[ 6930.840157] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) Skipped 1 previous similar message
[ 6950.903056] Lustre: soaked-MDT0000: Received LWP connection from 192.168.1.104@o2ib10, removing former export from 192.168.1.105@o2ib10
[ 6956.871962] Lustre: soaked-MDT0000: Received LWP connection from 192.168.1.104@o2ib10, removing former export from 192.168.1.105@o2ib10
[ 6961.975059] Lustre: soaked-MDT0000: Received LWP connection from 192.168.1.104@o2ib10, removing former export from 192.168.1.105@o2ib10
[ 6967.630431] Lustre: soaked-MDT0000: Received LWP connection from 192.168.1.104@o2ib10, removing former export from 192.168.1.105@o2ib10
[ 7015.073630] LustreError: 11-0: soaked-OST0009-osc-MDT0000: operation ost_create to node 192.168.1.104@o2ib10 failed: rc = -19
[ 7015.089895] Lustre: soaked-OST0009-osc-MDT0000: Connection to soaked-OST0009 (at 192.168.1.104@o2ib10) was lost; in progress operations using this service will wait for recovery to complete
[ 7015.115197] LustreError: 3577:0:(osp_precreate.c:602:osp_precreate_send()) soaked-OST0009-osc-MDT0000: can't precreate: rc = -19
[ 7015.134131] LustreError: 3577:0:(osp_precreate.c:1257:osp_precreate_thread()) soaked-OST0009-osc-MDT0000: cannot precreate objects: rc = -19
[ 7065.802724] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for sent delay: [sent 1494360412/real 0] req@ffff8803f037d700 x1566943440507120/t0(0) o8->soaked-OST0009-osc-MDT0000@192.168.1.105@o2ib10:28/4 lens 520/544 e 0 to 1 dl 1494360418 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
[ 7065.844563] Lustre: 3457:0:(client.c:2114:ptlrpc_expire_one_request()) Skipped 3 previous similar messages
[ 7104.026447] LustreError: 11-0: soaked-OST0015-osc-MDT0000: operation ost_statfs to node 192.168.1.104@o2ib10 failed: rc = -107
[ 7104.046785] Lustre: soaked-OST0015-osc-MDT0000: Connection to soaked-OST0015 (at 192.168.1.104@o2ib10) was lost; in progress operations using this service will wait for recovery to complete
[ 7113.338839] LustreError: 11-0: soaked-OST0003-osc-MDT0000: operation ost_statfs to node 192.168.1.104@o2ib10 failed: rc = -107
[ 7113.359126] Lustre: soaked-OST0003-osc-MDT0000: Connection to soaked-OST0003 (at 192.168.1.104@o2ib10) was lost; in progress operations using this service will wait for recovery to complete
[ 7149.548919] LustreError: 11-0: soaked-OST000f-osc-MDT0000: operation ost_statfs to node 192.168.1.104@o2ib10 failed: rc = -107
[ 7149.565481] Lustre: soaked-OST000f-osc-MDT0000: Connection to soaked-OST000f (at 192.168.1.104@o2ib10) was lost; in progress operations using this service will wait for recovery to complete
[ 7166.616620] LustreError: 5550:0:(lfsck_lib.c:2680:lfsck_load_one_trace_file()) soaked-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_00: rc = 0
[ 7408.971576] NMI watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [OI_scrub:5551]
[ 7408.981437] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7409.118481] CPU: 6 PID: 5551 Comm: OI_scrub Tainted: P OE ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7409.132130] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7409.149309] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7409.159160] RIP: 0010:[<ffffffffa121d1d9>] [<ffffffffa121d1d9>] osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[ 7409.171858] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[ 7409.179266] RAX: 0000000000000004 RBX: 0000000023f30a01 RCX: 0000000000000000
[ 7409.188702] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7409.198119] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 7409.207516] R10: 0000000023f30a01 R11: ffffea000c8fcc00 R12: 0000000023f30a01
[ 7409.216888] R13: ffff880703603d08 R14: 0000000023f30a01 R15: ffff880703603d08
[ 7409.226244] FS: 0000000000000000(0000) GS:ffff88042e180000(0000) knlGS:0000000000000000
[ 7409.236663] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7409.244443] CR2: 00007f64d55202e0 CR3: 00000000019be000 CR4: 00000000000407e0
[ 7409.253766] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7409.263065] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7409.272335] Stack:
[ 7409.275859] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 7409.285442] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 7409.294988] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 7409.304513] Call Trace:
[ 7409.308465] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7409.317858] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7409.326938] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7409.335799] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7409.343188] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7409.352696] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7409.359268] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7409.367669] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7409.374792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7409.383151] Code: 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5
[ 7449.101981] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7449.111641] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7449.244273] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7449.257593] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7449.273896] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7449.283300] RIP: 0010:[<ffffffffa121cfe9>] [<ffffffffa121cfe9>] osd_inode_iteration+0x299/0xcc0 [osd_ldiskfs]
[ 7449.295582] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[ 7449.302543] RAX: ffff88042bd78800 RBX: ffff880703603d57 RCX: ffff880703603d98
[ 7449.311540] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff88042bd79800
[ 7449.320527] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 7449.329498] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 7449.338460] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[ 7449.347419] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7449.357453] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7449.364869] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7449.373845] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7449.382821] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7449.391781] Stack:
[ 7449.394999] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 7449.404292] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 7449.413581] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 7449.422887] Call Trace:
[ 7449.426631] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7449.435796] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7449.444675] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7449.453365] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7449.460583] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7449.469952] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7449.476389] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7449.484672] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7449.491690] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7449.499965] Code: 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89
[ 7477.102955] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7477.112593] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7477.244786] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7477.258109] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7477.274414] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7477.283822] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[ 7477.295506] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 7477.302474] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[ 7477.311475] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7477.320458] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7477.329426] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 7477.338386] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 7477.347339] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7477.357380] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7477.364792] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7477.373764] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7477.382733] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7477.391685] Stack:
[ 7477.394896] ffff880703603d08 ffff880703603d57 ffff880703603df0 000000009f8f2001
[ 7477.404190] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 7477.413486] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[ 7477.422765] Call Trace:
[ 7477.426502] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7477.435674] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7477.444848] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7477.453722] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7477.462408] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7477.469617] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7477.478970] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7477.485408] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7477.493700] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7477.500712] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7477.508990] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[ 7483.585176] INFO: rcu_sched self-detected stall on CPU { 24} (t=60000 jiffies g=907112 c=907111 q=32519)
[ 7483.597097] Task dump for CPU 24:
[ 7483.601835] OI_scrub R running task 0 5551 2 0x00000088
[ 7483.610750] INFO: rcu_sched detected stalls on CPUs/tasks: {
[ 7483.610750] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 7483.627368] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 7483.636644] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 7483.645897] Call Trace:
[ 7483.649561] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 7483.657516] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 7483.664455] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 7483.671954] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 7483.679627] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 7483.687875] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 7483.695408] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 7483.703412] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 7483.710514] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 7483.718081] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 7483.725353] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 7483.732218] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 7483.740259] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 7483.748085] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 7483.755511] <EOI> [<ffffffffa121d1fc>] ? osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[ 7483.765316] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7483.774416] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7483.783334] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7483.791961] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7483.800373] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7483.807324] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7483.816412] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7483.822603] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7483.830628] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7483.837381] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7483.845401] 24} (detected by 9, t=60262 jiffies, g=907112, c=907111, q=32606)
[ 7483.854251] Task dump for CPU 24:
[ 7483.858683] OI_scrub R running task 0 5551 2 0x00000088
[ 7483.867293] ffff880703603640 ffffffff8168bdc0 ffff88083fde6dd0 ffff880703603fd8
[ 7483.876334] ffff880703603fd8 ffff880703603fd8 ffff88083fde6dd0 ffff88082da16c40
[ 7483.885368] 0000000000000000 7fffffffffffffff ffff88043ffac170 ffffffff8168a440
[ 7483.894398] Call Trace:
[ 7483.897867] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[ 7483.904734] [<ffffffff8168a440>] ? bit_wait+0x50/0x50
[ 7483.911197] [<ffffffff8168c3c9>] schedule+0x29/0x70
[ 7483.917460] [<ffffffff81689e29>] schedule_timeout+0x239/0x2c0
[ 7483.924705] [<ffffffff810eb08c>] ? ktime_get_ts64+0x4c/0xf0
[ 7483.931744] [<ffffffff8113d19f>] ? delayacct_end+0x8f/0xb0
[ 7483.938663] [<ffffffff8113d2c4>] ? __delayacct_blkio_end+0x34/0x60
[ 7483.946341] [<ffffffff8168b9a8>] io_schedule_timeout+0xe8/0x130
[ 7483.953708] [<ffffffff81689f8e>] ? __wait_on_bit+0x7e/0x90
[ 7483.960586] [<ffffffff8168a440>] ? bit_wait+0x50/0x50
[ 7483.966974] [<ffffffffa120add0>] ? iam_lfix_split+0x140/0x140 [osd_ldiskfs]
[ 7483.975489] [<ffffffff8168a021>] ? out_of_line_wait_on_bit+0x81/0xb0
[ 7483.983318] [<ffffffff810b1be0>] ? wake_bit_function+0x40/0x40
[ 7483.990568] [<ffffffff8123339a>] ? __wait_on_buffer+0x2a/0x30
[ 7483.997734] [<ffffffffa1178c9c>] ? ldiskfs_bread+0x7c/0xc0 [ldiskfs]
[ 7484.005577] [<ffffffffa11734aa>] ? ldiskfs_es_lookup_extent+0x2a/0x180 [ldiskfs]
[ 7484.014572] [<ffffffffa11734aa>] ? ldiskfs_es_lookup_extent+0x2a/0x180 [ldiskfs]
[ 7484.023547] [<ffffffff8118feb9>] ? mark_page_accessed+0x59/0xa0
[ 7484.030866] [<ffffffff8123366c>] ? __find_get_block+0xbc/0x120
[ 7484.038089] [<ffffffff812336fd>] ? __getblk+0x2d/0x300
[ 7484.044541] [<ffffffffa1178ac6>] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs]
[ 7484.052550] [<ffffffffa1178c47>] ? ldiskfs_bread+0x27/0xc0 [ldiskfs]
[ 7484.060360] [<ffffffffa120a7cd>] ? iam_lfix_lookup+0x19d/0x3b0 [osd_ldiskfs]
[ 7484.068996] [<ffffffffa0cb56c1>] ? lprocfs_counter_sub+0xc1/0x130 [obdclass]
[ 7484.077593] [<ffffffff81232b2d>] ? __brelse+0x3d/0x50
[ 7484.083960] [<ffffffff810bc454>] ? __wake_up+0x44/0x50
[ 7484.090422] [<ffffffffa026137a>] ? jbd2_journal_stop+0x1ea/0x3d0 [jbd2]
[ 7484.098538] [<ffffffffa12038e4>] ? osd_oi_lookup+0xa4/0x160 [osd_ldiskfs]
[ 7484.106846] [<ffffffff811dc861>] ? __slab_free+0x81/0x2f0
[ 7484.113602] [<ffffffffa112911c>] ? __ldiskfs_journal_stop+0x3c/0xb0 [ldiskfs]
[ 7484.122299] [<ffffffffa12173a5>] ? osd_scrub_refresh_mapping+0x1b5/0x440 [osd_ldiskfs]
[ 7484.131845] [<ffffffffa12184d1>] ? osd_scrub_check_update+0x4d1/0x10c0 [osd_ldiskfs]
[ 7484.141208] [<ffffffff811dd173>] ? kfree+0x103/0x140
[ 7484.147472] [<ffffffffa12184d1>] ? osd_scrub_check_update+0x4d1/0x10c0 [osd_ldiskfs]
[ 7484.156840] [<ffffffffa121ba09>] ? osd_scrub_exec+0x79/0x5b0 [osd_ldiskfs]
[ 7484.165246] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7484.174236] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7484.183014] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7484.191503] [<ffffffffa121e370>] ? osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7484.199983] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7484.206821] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7484.215791] [<ffffffff810b0a4f>] ? kthread+0xcf/0xe0
[ 7484.222021] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7484.229908] [<ffffffff81697318>] ? ret_from_fork+0x58/0x90
[ 7484.236718] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7509.104065] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7509.113369] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7509.242314] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7509.255409] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7509.271320] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7509.280561] RIP: 0010:[<ffffffffa121ba1d>] [<ffffffffa121ba1d>] osd_scrub_exec+0x8d/0x5b0 [osd_ldiskfs]
[ 7509.292090] RSP: 0018:ffff880703603ce8 EFLAGS: 00000292
[ 7509.298925] RAX: 0000000000000000 RBX: ffff880703603c78 RCX: 0000000000000000
[ 7509.307803] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7509.316693] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7509.325591] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[ 7509.334485] R13: ffffffffa121b9c4 R14: ffffffffffffff10 R15: ffff8803162f6000
[ 7509.343378] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7509.353346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7509.360693] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7509.369618] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7509.378521] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7509.387406] Stack:
[ 7509.390565] ffff8800b2a3746c 0000000000000004 00000000ffffffff 0000000000000001
[ 7509.399800] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[ 7509.409057] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 7509.418311] Call Trace:
[ 7509.421999] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7509.431126] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7509.440249] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7509.449073] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7509.457701] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7509.464870] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7509.474188] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7509.480577] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7509.488817] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7509.495804] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7509.504051] Code: 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b <41> 5c 41 5d 41 5e 41 5f 5d c3 66 0f 1f 84 00 00 00 00 00 4c 8d
[ 7537.105047] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7537.114663] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7537.247015] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7537.260328] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7537.276595] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7537.286000] RIP: 0010:[<ffffffffa121ba1d>] [<ffffffffa121ba1d>] osd_scrub_exec+0x8d/0x5b0 [osd_ldiskfs]
[ 7537.297674] RSP: 0018:ffff880703603ce8 EFLAGS: 00000292
[ 7537.304628] RAX: 0000000000000000 RBX: ffff880703603c78 RCX: 0000000000000000
[ 7537.313620] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7537.322600] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7537.331561] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[ 7537.340513] R13: ffffffffa121ba87 R14: ffffffffffffff10 R15: ffff8803162f6000
[ 7537.349467] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7537.359492] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7537.366905] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7537.375875] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7537.384843] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7537.393790] Stack:
[ 7537.397004] ffff8800b2a3746c 0000000000000004 00000000ffffffff 0000000000000001
[ 7537.406306] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[ 7537.415609] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 7537.424905] Call Trace:
[ 7537.428632] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7537.437797] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7537.446967] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7537.455835] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7537.464502] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7537.471716] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7537.481075] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7537.487515] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7537.495819] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7537.502832] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7537.511119] Code: 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b <41> 5c 41 5d 41 5e 41 5f 5d c3 66 0f 1f 84 00 00 00 00 00 4c 8d
[ 7565.106022] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7565.115664] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7565.247902] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7565.261189] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7565.277449] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7565.286832] RIP: 0010:[<ffffffffa1217a25>] [<ffffffffa1217a25>] osd_scrub_next+0x5/0x3c0 [osd_ldiskfs]
[ 7565.298396] RSP: 0018:ffff880703603d10 EFLAGS: 00000287
[ 7565.305342] RAX: ffffffffa1217a20 RBX: ffff880703603d57 RCX: ffff880703603d98
[ 7565.314325] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7565.323293] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 7565.332256] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 7565.341209] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[ 7565.350164] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7565.360200] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7565.367615] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7565.376596] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7565.385560] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7565.394512] Stack:
[ 7565.397729] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 7565.407010] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 7565.416320] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 7565.425624] Call Trace:
[ 7565.429352] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 7565.438720] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7565.447892] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7565.456767] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7565.465434] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7565.472655] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7565.482018] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7565.488453] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7565.496748] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7565.503768] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7565.512041] Code: 48 c1 e0 03 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 <55> 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4
[ 7593.106996] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7593.116636] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7593.248385] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7593.261703] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7593.277982] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7593.287378] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[ 7593.299615] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[ 7593.306576] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 7593.315559] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7593.324539] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 7593.333514] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 7593.342481] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 7593.351449] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7593.361494] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7593.368919] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7593.377902] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7593.386867] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7593.395820] Stack:
[ 7593.399038] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 7593.408348] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 7593.417638] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 7593.426928] Call Trace:
[ 7593.430651] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7593.439824] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7593.448702] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7593.457373] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7593.464586] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7593.473935] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7593.480371] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7593.488652] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7593.495668] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7593.503944] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[ 7621.107972] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7621.117616] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7621.250206] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7621.263532] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7621.279855] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7621.289275] RIP: 0010:[<ffffffffa121d1c3>] [<ffffffffa121d1c3>] osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[ 7621.301566] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[ 7621.308545] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[ 7621.317539] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 7621.326542] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 7621.335515] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 7621.344468] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 7621.353425] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7621.363457] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7621.370871] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7621.379849] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7621.388817] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7621.397769] Stack:
[ 7621.400987] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 7621.410288] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 7621.419586] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 7621.428890] Call Trace:
[ 7621.432623] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7621.441778] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7621.450655] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7621.459344] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7621.466561] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7621.475914] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7621.482350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7621.490652] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7621.497669] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7621.505952] Code: 00 00 48 c7 05 9b 86 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 <4c> 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff
[ 7649.108955] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7649.118578] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7649.250555] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7649.263860] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7649.280148] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7649.289550] RIP: 0010:[<ffffffffa121b995>] [<ffffffffa121b995>] osd_scrub_exec+0x5/0x5b0 [osd_ldiskfs]
[ 7649.301104] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[ 7649.308063] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 7649.317046] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7649.326028] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 7649.334988] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 7649.343936] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 7649.352885] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7649.362911] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7649.370319] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7649.379293] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7649.388258] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7649.397215] Stack:
[ 7649.400438] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 7649.409730] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 7649.419029] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 7649.428334] Call Trace:
[ 7649.432069] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7649.441434] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7649.450606] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7649.459492] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7649.468169] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7649.475390] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7649.484744] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7649.491187] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7649.499478] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7649.506495] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7649.514770] Code: 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 <55> 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83
[ 7663.615454] INFO: rcu_sched self-detected stall on CPU { 24} (t=240024 jiffies g=907112 c=907111 q=317538)
[ 7663.627634] Task dump for CPU 24:
[ 7663.632386] OI_scrub R running task 0 5551 2 0x00000088
[ 7663.641323] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 7663.650653] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 7663.659955] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 7663.669240] Call Trace:
[ 7663.672935] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 7663.680934] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 7663.687912] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 7663.695447] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 7663.703157] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 7663.711421] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 7663.718983] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 7663.727007] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 7663.734156] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 7663.741759] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 7663.749052] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 7663.755938] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 7663.763976] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 7663.771824] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 7663.779260] <EOI> [<ffffffffa1217b69>] ? osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[ 7663.788604] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 7663.797522] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7663.806424] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7663.815043] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7663.823465] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7663.830419] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7663.839515] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7663.845691] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7663.853721] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7663.860482] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7689.110349] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 7689.119772] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7689.249982] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7689.263158] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7689.279206] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7689.288508] RIP: 0010:[<ffffffffa121d00e>] [<ffffffffa121d00e>] osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[ 7689.300660] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[ 7689.307516] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[ 7689.316415] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 7689.325307] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 7689.334197] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 7689.343091] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 7689.351979] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7689.361941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7689.369286] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7689.378188] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7689.387095] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7689.395988] Stack:
[ 7689.399152] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 7689.408394] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 7689.417632] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 7689.426898] Call Trace:
[ 7689.430592] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7689.439729] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7689.448569] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7689.457213] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7689.464402] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7689.473722] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7689.480118] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7689.488366] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7689.495347] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7689.503597] Code: 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 <66> 90 0f 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90
[ 7717.111318] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 7717.120935] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7717.253424] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7717.266748] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7717.283101] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7717.292534] RIP: 0010:[<ffffffffa1162b36>] [<ffffffffa1162b36>] ldiskfs_itable_unused_count+0x6/0x30 [ldiskfs]
[ 7717.304894] RSP: 0018:ffff880703603d08 EFLAGS: 00000286
[ 7717.311879] RAX: ffff8800b2a37468 RBX: ffff880703603d57 RCX: 0000000000000000
[ 7717.320879] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 7717.329874] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 7717.338848] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 7717.347809] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[ 7717.356756] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7717.366776] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7717.374181] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7717.383155] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7717.392119] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7717.401065] Stack:
[ 7717.404272] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[ 7717.413548] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 7717.422845] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 7717.432137] Call Trace:
[ 7717.435883] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 7717.445239] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7717.454414] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7717.463280] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7717.471962] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7717.479173] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7717.488536] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7717.494963] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7717.503238] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7717.510245] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7717.518512] Code: 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 <48> 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76
[ 7745.112287] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 7745.121947] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7745.254065] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7745.267351] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7745.283624] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7745.293014] RIP: 0010:[<ffffffffa1162b55>] [<ffffffffa1162b55>] ldiskfs_itable_unused_count+0x25/0x30 [ldiskfs]
[ 7745.305452] RSP: 0018:ffff880703603d08 EFLAGS: 00000202
[ 7745.312393] RAX: 00000000000072ce RBX: 0000000000000001 RCX: ffff88042bd78800
[ 7745.321391] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 7745.330368] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7745.339313] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 7745.348262] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[ 7745.357211] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7745.367235] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7745.374639] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7745.383605] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7745.392568] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7745.401514] Stack:
[ 7745.404718] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[ 7745.414005] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 7745.423313] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 7745.432596] Call Trace:
[ 7745.436323] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 7745.445503] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7745.454666] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7745.463526] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7745.472198] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7745.479398] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7745.488764] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7745.495186] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7745.503460] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7745.510467] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7745.518736] Code: 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76 07 0f b7 46 32 c1 e0 10 09 d0 <5d> c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 89 16 48 8b
[ 7773.113267] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 7773.122905] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7773.255132] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7773.268412] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7773.284667] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7773.294051] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[ 7773.305698] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[ 7773.312640] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[ 7773.321605] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7773.330566] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7773.339514] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 7773.348459] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 7773.357409] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7773.367435] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7773.374842] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7773.383816] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7773.392779] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7773.401735] Stack:
[ 7773.404951] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[ 7773.414264] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 7773.423561] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[ 7773.432850] Call Trace:
[ 7773.436576] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7773.445743] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7773.454903] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7773.463771] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7773.472458] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7773.479664] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7773.489025] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7773.495453] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7773.503753] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7773.510765] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7773.519034] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[ 7801.114237] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7801.123938] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7801.256247] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7801.269533] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7801.285798] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7801.295184] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[ 7801.306838] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 7801.313785] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[ 7801.322770] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7801.331741] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7801.340702] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 7801.349659] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 7801.358617] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7801.368652] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7801.376065] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7801.385029] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7801.393991] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7801.402940] Stack:
[ 7801.406156] ffff880703603d08 0000000000000001 ffff880703603df0 000000009f8f2001
[ 7801.415463] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 7801.424760] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[ 7801.434050] Call Trace:
[ 7801.437771] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7801.446947] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7801.456108] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7801.464971] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7801.473643] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7801.480851] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7801.490206] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7801.496636] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7801.504929] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7801.511939] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7801.520215] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[ 7829.115219] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7829.124851] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7829.257016] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7829.270298] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7829.286560] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7829.295930] RIP: 0010:[<ffffffffa1162b53>] [<ffffffffa1162b53>] ldiskfs_itable_unused_count+0x23/0x30 [ldiskfs]
[ 7829.308364] RSP: 0018:ffff880703603d08 EFLAGS: 00000297
[ 7829.315308] RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff88042bd78800
[ 7829.324289] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 7829.333248] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7829.342204] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 7829.351156] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[ 7829.360105] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7829.370132] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7829.377534] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7829.386503] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7829.395453] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7829.404403] Stack:
[ 7829.407607] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[ 7829.416904] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 7829.426195] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 7829.435482] Call Trace:
[ 7829.439215] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 7829.448378] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7829.457538] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7829.466399] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7829.475074] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7829.482282] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7829.491637] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7829.498066] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7829.506354] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7829.513362] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7829.521629] Code: 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76 07 0f b7 46 32 c1 e0 10 <09> d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 89 16
[ 7843.624724] INFO: rcu_sched self-detected stall on CPU { 24} (t=420027 jiffies g=907112 c=907111 q=736665)
[ 7843.636869] Task dump for CPU 24:
[ 7843.641610] OI_scrub R running task 0 5551 2 0x00000088
[ 7843.650505] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 7843.659825] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 7843.669131] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 7843.678406] Call Trace:
[ 7843.682093] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 7843.690070] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 7843.697024] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 7843.704546] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 7843.712237] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 7843.720488] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 7843.728035] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 7843.736038] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 7843.743154] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 7843.750729] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 7843.758002] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 7843.764881] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 7843.772900] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 7843.780721] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 7843.788142] <EOI> [<ffffffffa121d1eb>] ? osd_inode_iteration+0x49b/0xcc0 [osd_ldiskfs]
[ 7843.797973] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7843.807065] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7843.815959] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7843.824566] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7843.832979] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7843.839935] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7843.849048] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7843.855225] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7843.863269] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7843.870032] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7869.116620] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7869.126049] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7869.256223] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7869.269396] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7869.285484] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7869.294797] RIP: 0010:[<ffffffffa121ba64>] [<ffffffffa121ba64>] osd_scrub_exec+0xd4/0x5b0 [osd_ldiskfs]
[ 7869.306380] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[ 7869.313247] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[ 7869.322158] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7869.331063] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7869.339966] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 7869.348863] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 7869.357763] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7869.367738] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7869.375092] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7869.384003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7869.392922] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7869.401820] Stack:
[ 7869.405003] 0000000000000004 0000000000000001 0000000000000004 ffff880703603d98
[ 7869.414269] ffff880703603d57 0000000000000000 ffff8803162f6000 ffffffffffffff10
[ 7869.423526] ffffffffa1217b80 0000000000000010 0000000000000282 ffff880703603ce0
[ 7869.432774] Call Trace:
[ 7869.436482] [<ffffffffa1217b80>] ? osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[ 7869.445332] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7869.454476] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7869.463614] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7869.472457] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7869.481104] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7869.488277] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7869.497603] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7869.504005] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7869.512255] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7869.519240] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7869.527491] Code: 4d 88 4c 89 ef e8 1d fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 <f6> 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00
[ 7888.786761] perf: interrupt took too long (2552 > 2500), lowering kernel.perf_event_max_sample_rate to 78000
[ 7913.118138] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7913.127746] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7913.259299] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7913.272570] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7913.288783] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7913.298151] RIP: 0010:[<ffffffffa121d1eb>] [<ffffffffa121d1eb>] osd_inode_iteration+0x49b/0xcc0 [osd_ldiskfs]
[ 7913.310371] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[ 7913.317309] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 7913.326285] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7913.335257] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 7913.344228] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 7913.353209] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffffea00027e3c80
[ 7913.362181] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7913.372233] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7913.379655] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7913.388634] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7913.397619] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7913.406591] Stack:
[ 7913.409834] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 7913.419151] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 7913.428463] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 7913.437783] Call Trace:
[ 7913.441527] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7913.450708] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7913.459594] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7913.468289] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7913.475524] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7913.484895] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7913.491358] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7913.499671] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7913.506723] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7913.515022] Code: 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 <41> 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 48 8b 85 50
[ 7941.119120] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7941.128790] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7941.261388] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7941.274706] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7941.291031] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7941.300454] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[ 7941.312154] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 7941.319131] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[ 7941.328130] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7941.337109] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7941.346080] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 7941.355044] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 7941.363995] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7941.374031] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7941.381447] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7941.390422] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7941.399400] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7941.408369] Stack:
[ 7941.411588] ffff880703603df0 000000009f8f2001 ffffea00027e3c80 000000009f8f2001
[ 7941.420880] ffff880703603d57 ffff880703603d57 ffff8800b2a37468 0000000000000000
[ 7941.430169] ffff880703603d78 ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[ 7941.439465] Call Trace:
[ 7941.443198] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7941.452369] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7941.461526] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7941.470398] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7941.479076] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7941.486284] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7941.495639] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7941.502069] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7941.510361] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7941.517353] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7941.525623] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[ 7969.120090] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7969.129754] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7969.261862] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7969.275159] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7969.291410] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7969.300802] RIP: 0010:[<ffffffffa1162b53>] [<ffffffffa1162b53>] ldiskfs_itable_unused_count+0x23/0x30 [ldiskfs]
[ 7969.313246] RSP: 0018:ffff880703603d08 EFLAGS: 00000297
[ 7969.320196] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[ 7969.329171] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 7969.338135] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7969.347081] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 7969.356025] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[ 7969.364974] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7969.375000] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7969.382402] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7969.391364] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7969.400320] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7969.409266] Stack:
[ 7969.412470] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[ 7969.421746] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 7969.431025] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 7969.440317] Call Trace:
[ 7969.444049] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 7969.453216] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7969.462380] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7969.471244] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7969.479914] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7969.487122] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7969.496477] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7969.502915] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7969.511194] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7969.518206] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7969.526484] Code: 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76 07 0f b7 46 32 c1 e0 10 <09> d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 89 16
[ 7997.121077] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 7997.130716] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 7997.262935] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 7997.276226] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 7997.292496] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 7997.301883] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[ 7997.313547] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 7997.320483] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[ 7997.329454] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 7997.338418] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 7997.347370] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 7997.356319] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 7997.365272] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 7997.375303] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7997.382713] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 7997.391689] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7997.400652] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7997.409611] Stack:
[ 7997.412828] ffff880703603d08 000000009f8f2001 ffff880703603df0 000000009f8f2001
[ 7997.422139] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 7997.431433] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[ 7997.440727] Call Trace:
[ 7997.444457] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 7997.453626] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 7997.462781] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 7997.471647] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 7997.480318] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 7997.487523] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 7997.496879] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 7997.503318] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7997.511605] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 7997.518619] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 7997.526891] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[ 8023.633993] INFO: rcu_sched self-detected stall on CPU { 24} (t=600030 jiffies g=907112 c=907111 q=1108460)
[ 8023.646258] Task dump for CPU 24:
[ 8023.651005] OI_scrub R running task 0 5551 2 0x00000088
[ 8023.659926] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 8023.669234] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 8023.678525] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 8023.687789] Call Trace:
[ 8023.691478] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 8023.699452] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 8023.706417] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 8023.713942] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 8023.721633] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 8023.729891] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 8023.737443] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 8023.745457] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 8023.752577] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 8023.760163] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 8023.767445] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 8023.774323] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 8023.782349] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 8023.790177] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 8023.797608] <EOI> [<ffffffffa121cfa1>] ? osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[ 8023.807445] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8023.816543] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8023.825451] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8023.834068] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8023.842480] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8023.849433] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8023.858526] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8023.864699] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8023.872740] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8023.879503] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8049.122881] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8049.132267] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8049.262514] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8049.275697] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8049.291782] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8049.301086] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[ 8049.313241] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[ 8049.320113] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8049.329026] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8049.337939] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8049.346837] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8049.355745] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8049.364650] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8049.374629] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8049.381983] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8049.390898] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8049.399819] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8049.408727] Stack:
[ 8049.411904] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8049.421167] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8049.430440] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8049.439683] Call Trace:
[ 8049.443369] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8049.452515] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8049.461364] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8049.470030] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8049.477234] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8049.486571] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8049.492991] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8049.501251] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8049.508242] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8049.516501] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[ 8077.123851] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8077.133459] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8077.265336] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8077.278659] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8077.294975] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8077.304377] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[ 8077.316632] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[ 8077.323601] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8077.332597] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8077.341584] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8077.350554] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8077.359514] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8077.368469] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8077.378489] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8077.385899] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8077.394869] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8077.403834] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8077.412792] Stack:
[ 8077.416010] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8077.425305] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8077.434596] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8077.443892] Call Trace:
[ 8077.447602] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8077.456762] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8077.465637] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8077.474315] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8077.481537] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8077.490884] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8077.497317] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8077.505602] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8077.512624] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8077.520909] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[ 8098.494062] perf: interrupt took too long (3223 > 3190), lowering kernel.perf_event_max_sample_rate to 62000
[ 8125.125524] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8125.135792] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8125.268115] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8125.281439] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8125.297751] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8125.307192] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[ 8125.318888] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[ 8125.325856] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[ 8125.334860] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8125.343854] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[ 8125.352832] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 8125.361800] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 8125.370776] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8125.380831] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8125.388267] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8125.397270] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8125.406258] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8125.415236] Stack:
[ 8125.418468] 000000009f8f2001 0000000000000004 ffff880703603d57 ffff880703603df0
[ 8125.427784] 0000000000000000 ffff880703603d57 000000009f8f2001 0000000000000004
[ 8125.437107] ffff880703603d57 ffff8800b2a37468 0000000000000000 ffff880703603d78
[ 8125.446420] Call Trace:
[ 8125.450181] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8125.459371] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8125.468559] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8125.477448] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8125.486150] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8125.493401] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8125.502783] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8125.509234] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8125.517554] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8125.524591] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8125.532893] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[ 8153.126499] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8153.136264] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8153.268064] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8153.281369] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8153.297634] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8153.307031] RIP: 0010:[<ffffffffa121d1d9>] [<ffffffffa121d1d9>] osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[ 8153.319285] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[ 8153.326240] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8153.335674] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8153.344653] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8153.353629] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8153.362589] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffff880703603d08
[ 8153.371569] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8153.381612] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8153.389025] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8153.398000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8153.406970] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8153.415932] Stack:
[ 8153.419151] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8153.428437] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8153.438223] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8153.447535] Call Trace:
[ 8153.451257] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8153.460445] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8153.469329] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8153.478003] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8153.485218] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8153.494578] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8153.501010] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8153.509315] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8153.516342] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8153.524626] Code: 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5
[ 8181.127481] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8181.137147] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8181.269565] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8181.282882] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8181.299193] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8181.308618] RIP: 0010:[<ffffffffa1162b53>] [<ffffffffa1162b53>] ldiskfs_itable_unused_count+0x23/0x30 [ldiskfs]
[ 8181.321095] RSP: 0018:ffff880703603d08 EFLAGS: 00000297
[ 8181.328069] RAX: 0000000000000000 RBX: ffff880703603d57 RCX: ffff88042bd78800
[ 8181.337067] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 8181.346039] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 8181.355013] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8181.363975] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[ 8181.372924] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8181.382952] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8181.390353] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8181.399321] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8181.408282] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8181.417236] Stack:
[ 8181.420449] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[ 8181.429733] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 8181.439010] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 8181.448294] Call Trace:
[ 8181.452052] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 8181.461219] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8181.470374] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8181.479245] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8181.487926] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8181.495131] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8181.504482] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8181.510896] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8181.519178] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8181.526186] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8181.534454] Code: 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76 07 0f b7 46 32 c1 e0 10 <09> d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 89 16
[ 8203.643258] INFO: rcu_sched self-detected stall on CPU { 24} (t=780033 jiffies g=907112 c=907111 q=1271641)
[ 8203.655527] Task dump for CPU 24:
[ 8203.660271] OI_scrub R running task 0 5551 2 0x00000088
[ 8203.669205] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 8203.678539] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 8203.687824] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 8203.697102] Call Trace:
[ 8203.700774] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 8203.708760] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 8203.715715] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 8203.723229] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 8203.730917] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 8203.739167] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 8203.746742] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 8203.754764] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 8203.761890] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 8203.769481] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 8203.776750] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 8203.783629] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 8203.791651] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 8203.799484] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 8203.806914] <EOI> [<ffffffffa1217a4b>] ? osd_scrub_next+0x2b/0x3c0 [osd_ldiskfs]
[ 8203.816164] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8203.825059] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8203.833950] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8203.842558] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8203.850980] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8203.857929] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8203.867035] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8203.873205] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8203.881233] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8203.887979] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8229.129152] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8229.138572] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8229.268671] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8229.281835] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8229.297898] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8229.307193] RIP: 0010:[<ffffffffa121d1fc>] [<ffffffffa121d1fc>] osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[ 8229.319328] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[ 8229.326188] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8229.335085] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8229.343987] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8229.352880] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 8229.361779] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8229.370669] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8229.380626] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8229.387968] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8229.396870] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8229.405778] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8229.414673] Stack:
[ 8229.417837] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8229.427082] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8229.436319] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8229.445556] Call Trace:
[ 8229.449253] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8229.458386] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8229.467228] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8229.475889] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8229.483079] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8229.492414] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8229.498813] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8229.507081] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8229.514072] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8229.522325] Code: 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 <48> 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2
[ 8257.130127] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8257.139734] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8257.271672] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8257.284980] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8257.301298] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8257.310697] RIP: 0010:[<ffffffffa121b990>] [<ffffffffa121b990>] osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8257.322765] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[ 8257.329727] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8257.338709] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8257.347699] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8257.356665] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 8257.365622] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8257.374576] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8257.384603] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8257.392014] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8257.400984] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8257.409954] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8257.418910] Stack:
[ 8257.422129] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 8257.431420] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 8257.440721] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 8257.450022] Call Trace:
[ 8257.453748] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8257.463119] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8257.472294] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8257.481177] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8257.489877] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8257.497096] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8257.506453] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8257.512895] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8257.521190] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8257.528218] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8257.536499] Code: b5 07 00 00 c7 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45
[ 8285.131096] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8285.140754] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8285.272515] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8285.285810] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8285.302089] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8285.311482] RIP: 0010:[<ffffffffa1217a4b>] [<ffffffffa1217a4b>] osd_scrub_next+0x2b/0x3c0 [osd_ldiskfs]
[ 8285.323144] RSP: 0018:ffff880703603c70 EFLAGS: 00000282
[ 8285.330091] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: ffff880703603d98
[ 8285.339082] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8285.348055] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[ 8285.357011] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 8285.365975] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 8285.374940] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8285.384982] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8285.392402] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8285.401380] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8285.410348] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8285.419303] Stack:
[ 8285.422508] 000000009f8f2001 ffff880703603d08 000000009f8f2001 ffff880703603df0
[ 8285.431812] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[ 8285.441117] ffff880703603d57 0000000000000000 0000000000000000 ffff880703603d78
[ 8285.450392] Call Trace:
[ 8285.454107] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8285.463273] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8285.472439] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8285.481307] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8285.489977] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8285.497186] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8285.506538] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8285.512969] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8285.521252] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8285.528270] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8285.536537] Code: 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87
[ 8313.132072] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8313.141708] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8313.273288] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8313.286569] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8313.302808] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8313.312195] RIP: 0010:[<ffffffffa1217b80>] [<ffffffffa1217b80>] osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[ 8313.323920] RSP: 0018:ffff880703603ce0 EFLAGS: 00000282
[ 8313.330861] RAX: 0000000000000004 RBX: ffff880703603c78 RCX: ffff880703603d98
[ 8313.339832] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 8313.348792] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[ 8313.357738] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[ 8313.366678] R13: ffffffffa121ba87 R14: ffffffffffffff10 R15: ffff8803162f6000
[ 8313.375619] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8313.385639] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8313.393033] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8313.401989] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8313.410933] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8313.419864] Stack:
[ 8313.423059] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[ 8313.432342] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[ 8313.441634] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[ 8313.450925] Call Trace:
[ 8313.454636] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8313.463801] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8313.472961] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8313.481821] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8313.490478] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8313.497684] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8313.507026] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8313.513453] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8313.521734] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8313.528749] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8313.537015] Code: 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0
[ 8341.133049] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8341.142675] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8341.274370] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8341.287660] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8341.303922] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8341.313306] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[ 8341.324941] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 8341.331881] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[ 8341.340857] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8341.349821] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 8341.358772] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 8341.367723] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 8341.376673] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8341.386710] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8341.394128] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8341.403100] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8341.412060] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8341.421008] Stack:
[ 8341.424221] ffff880703603d08 ffff880703603d57 ffff880703603df0 000000009f8f2001
[ 8341.433506] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 8341.442808] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[ 8341.452106] Call Trace:
[ 8341.455825] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8341.464997] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8341.474155] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8341.483022] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8341.491684] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8341.498890] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8341.508230] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8341.514664] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8341.522944] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8341.529952] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8341.538223] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[ 8369.134029] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8369.143658] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8369.275277] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8369.288566] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8369.304826] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8369.314206] RIP: 0010:[<ffffffffa121d010>] [<ffffffffa121d010>] osd_inode_iteration+0x2c0/0xcc0 [osd_ldiskfs]
[ 8369.326427] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[ 8369.333367] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[ 8369.342344] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 8369.351308] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 8369.360247] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 8369.369191] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8369.378140] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8369.388160] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8369.395563] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8369.404519] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8369.413465] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8369.422398] Stack:
[ 8369.425597] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8369.434878] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8369.444171] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8369.453463] Call Trace:
[ 8369.457161] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8369.466321] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8369.475183] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8369.483840] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8369.491036] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8369.500381] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8369.506806] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8369.515074] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8369.522087] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8369.530354] Code: 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 <0f> 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90 48 85
[ 8383.652544] INFO: rcu_sched self-detected stall on CPU { 24} (t=960036 jiffies g=907112 c=907111 q=1306848)
[ 8383.664808] Task dump for CPU 24:
[ 8383.669559] OI_scrub R running task 0 5551 2 0x00000088
[ 8383.678489] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 8383.687814] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 8383.697134] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 8383.706441] Call Trace:
[ 8383.710137] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 8383.718137] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 8383.725105] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 8383.732643] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 8383.740347] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 8383.748618] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 8383.756185] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 8383.764209] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 8383.771340] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 8383.778940] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 8383.786231] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 8383.793122] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 8383.801155] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 8383.808996] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 8383.816429] <EOI> [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8383.825750] [<ffffffffa1217a3e>] ? osd_scrub_next+0x1e/0x3c0 [osd_ldiskfs]
[ 8383.834272] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8383.843176] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8383.852080] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8383.860695] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8383.869103] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8383.876054] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8383.885138] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8383.891317] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8383.899340] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8383.906095] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8409.135416] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8409.144794] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8409.274556] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8409.287727] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8409.303769] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8409.313060] RIP: 0010:[<ffffffffa121ba26>] [<ffffffffa121ba26>] osd_scrub_exec+0x96/0x5b0 [osd_ldiskfs]
[ 8409.324621] RSP: 0018:ffff880703603d10 EFLAGS: 00000292
[ 8409.331490] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8409.340395] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8409.349292] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8409.358194] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 8409.367089] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8409.375967] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8409.385933] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8409.393278] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8409.402192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8409.411097] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8409.419988] Stack:
[ 8409.423155] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 8409.432404] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 8409.441649] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 8409.450901] Call Trace:
[ 8409.454593] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8409.463919] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8409.473049] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8409.481890] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8409.490538] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8409.497714] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8409.507046] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8409.513453] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8409.521700] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8409.528688] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8409.536937] Code: 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f 5d <c3> 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d 88
[ 8437.136398] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8437.145991] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8437.278124] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8437.291409] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8437.307676] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8437.317060] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[ 8437.329319] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[ 8437.336270] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8437.345259] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8437.354236] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8437.363191] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8437.372151] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8437.381107] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8437.391143] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8437.398553] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8437.407523] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8437.416502] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8437.425446] Stack:
[ 8437.428659] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8437.437927] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8437.447236] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8437.456535] Call Trace:
[ 8437.460272] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8437.469440] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8437.478320] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8437.486998] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8437.494212] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8437.503577] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8437.510015] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8437.518320] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8437.525340] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8437.533624] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[ 8460.520491] Lustre: MGS: Received new LWP connection from 192.168.1.124@o2ib100, removing former export from same NID
[ 8460.533571] Lustre: MGS: Connection restored to 5a0e6555-4c2d-39c9-90c4-a332f62c8353 (at 192.168.1.124@o2ib100)
[ 8460.545926] Lustre: Skipped 17 previous similar messages
[ 8465.137373] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8465.147021] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8465.279251] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8465.292541] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8465.308810] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8465.318211] RIP: 0010:[<ffffffffa121d1fc>] [<ffffffffa121d1fc>] osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[ 8465.330469] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[ 8465.337436] RAX: 0000000020000000 RBX: ffff880703603d57 RCX: 0000000000000000
[ 8465.346429] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8465.355425] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8465.364417] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8465.373406] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[ 8465.382403] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8465.392482] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8465.399926] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8465.408925] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8465.417907] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8465.426893] Stack:
[ 8465.430158] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8465.439494] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8465.448822] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8465.458147] Call Trace:
[ 8465.461913] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8465.471117] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8465.480024] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8465.488740] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8465.495993] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8465.505394] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8465.511868] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8465.520205] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8465.527257] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8465.535567] Code: 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 <48> 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2
[ 8493.138348] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8493.148023] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8493.280438] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8493.293742] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8493.310037] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8493.319449] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[ 8493.331709] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[ 8493.338673] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[ 8493.347664] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 8493.356647] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8493.365616] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8493.374574] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8493.383528] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8493.393557] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8493.400968] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8493.409947] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8493.418935] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8493.427912] Stack:
[ 8493.431133] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8493.440459] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8493.449765] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8493.459073] Call Trace:
[ 8493.462835] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8493.472036] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8493.480919] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8493.489610] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8493.496832] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8493.506200] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8493.512637] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8493.520925] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8493.527946] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8493.536233] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[ 8521.139318] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8521.148963] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8521.280892] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8521.294204] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8521.310529] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8521.319945] RIP: 0010:[<ffffffffa121ba09>] [<ffffffffa121ba09>] osd_scrub_exec+0x79/0x5b0 [osd_ldiskfs]
[ 8521.331617] RSP: 0018:ffff880703603c78 EFLAGS: 00000202
[ 8521.338595] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 00000000fb6db880
[ 8521.347592] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8521.356581] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 8521.365561] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 8521.374513] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 8521.383472] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8521.393509] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8521.400925] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8521.409900] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8521.418870] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8521.427827] Stack:
[ 8521.431048] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 0000000000000001
[ 8521.440342] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[ 8521.449630] ffffffffa1217a20 ffff880703603d98 ffff880703603d78 ffff8800b2a36000
[ 8521.458928] Call Trace:
[ 8521.462656] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8521.471545] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8521.480742] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8521.489927] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8521.498791] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8521.507459] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8521.514694] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8521.524057] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8521.530488] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8521.538778] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8521.545792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8521.554060] Code: 07 74 4f 41 83 f9 02 74 78 48 89 ca 44 89 c9 e8 0e c6 ff ff 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 <65> 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41
[ 8549.140299] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8549.149924] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8549.281524] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8549.294789] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8549.311005] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8549.320370] RIP: 0010:[<ffffffffa1162b36>] [<ffffffffa1162b36>] ldiskfs_itable_unused_count+0x6/0x30 [ldiskfs]
[ 8549.332691] RSP: 0018:ffff880703603d08 EFLAGS: 00000286
[ 8549.339637] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8549.348605] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 8549.357570] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8549.366529] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8549.375488] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000286
[ 8549.384440] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8549.394468] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8549.401867] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8549.410825] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8549.419776] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8549.428715] Stack:
[ 8549.431927] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[ 8549.441230] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 8549.450513] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 8549.459807] Call Trace:
[ 8549.463532] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 8549.472896] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8549.482060] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8549.490927] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8549.499584] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8549.506800] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8549.516151] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8549.522580] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8549.530863] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8549.537874] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8549.546151] Code: 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 <48> 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76
[ 8563.661807] INFO: rcu_sched self-detected stall on CPU { 24} (t=1140039 jiffies g=907112 c=907111 q=1391933)
[ 8563.674140] Task dump for CPU 24:
[ 8563.678865] OI_scrub R running task 0 5551 2 0x00000088
[ 8563.687782] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 8563.697105] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 8563.706407] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 8563.715688] Call Trace:
[ 8563.719368] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 8563.727339] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 8563.734293] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 8563.741813] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 8563.749507] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 8563.757757] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 8563.765305] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 8563.773307] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 8563.780430] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 8563.788014] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 8563.795284] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 8563.802157] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 8563.810179] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 8563.817996] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 8563.825414] <EOI> [<ffffffffa1217a4b>] ? osd_scrub_next+0x2b/0x3c0 [osd_ldiskfs]
[ 8563.834646] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8563.843544] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8563.852445] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8563.861051] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8563.869461] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8563.876414] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8563.885499] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8563.891664] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8563.899682] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8563.906430] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8589.141692] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8589.151085] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8589.280566] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8589.293731] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8589.309793] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8589.319081] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[ 8589.330636] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[ 8589.337486] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[ 8589.346389] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8589.355279] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[ 8589.364167] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 8589.373057] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 8589.381943] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8589.391904] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8589.399240] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8589.408136] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8589.417036] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8589.425927] Stack:
[ 8589.429087] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[ 8589.438309] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[ 8589.447558] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[ 8589.456798] Call Trace:
[ 8589.460461] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8589.469588] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8589.478709] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8589.487538] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8589.496174] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8589.503345] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8589.512645] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8589.519037] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8589.527280] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8589.534251] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8589.542486] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[ 8617.142669] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8617.152268] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8617.283823] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8617.297101] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8617.313343] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8617.322721] RIP: 0010:[<ffffffffa1162b46>] [<ffffffffa1162b46>] ldiskfs_itable_unused_count+0x16/0x30 [ldiskfs]
[ 8617.335139] RSP: 0018:ffff880703603d08 EFLAGS: 00000246
[ 8617.342074] RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff88042bd78800
[ 8617.351049] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 8617.360020] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 8617.368968] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8617.377912] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000286
[ 8617.386856] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8617.396872] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8617.404266] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8617.413217] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8617.422159] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8617.431097] Stack:
[ 8617.434291] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[ 8617.443573] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 8617.452859] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 8617.462135] Call Trace:
[ 8617.465853] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 8617.475022] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8617.484177] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8617.493039] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8617.501691] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8617.508898] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8617.518241] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8617.524664] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8617.532939] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8617.539944] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8617.548206] Code: 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 <48> 83 39 3f 76 07 0f b7 46 32 c1 e0 10 09 d0 5d c3 66 0f 1f 84
[ 8645.143643] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8645.153277] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8645.284989] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8645.298261] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8645.314518] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8645.323898] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[ 8645.336119] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[ 8645.343054] RAX: ffff88042bd78800 RBX: ffff880703603d57 RCX: ffff88042bd78800
[ 8645.352028] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 8645.360988] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8645.369934] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8645.378882] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[ 8645.387832] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8645.397862] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8645.405265] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8645.414231] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8645.423190] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8645.432145] Stack:
[ 8645.435357] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8645.444652] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8645.453970] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8645.463269] Call Trace:
[ 8645.466989] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8645.476157] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8645.485030] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8645.493711] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8645.500918] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8645.510282] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8645.516724] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8645.525015] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8645.532028] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8645.540303] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[ 8673.144612] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8673.154237] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8673.286111] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8673.299421] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8673.315735] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8673.325134] RIP: 0010:[<ffffffffa121ba26>] [<ffffffffa121ba26>] osd_scrub_exec+0x96/0x5b0 [osd_ldiskfs]
[ 8673.336804] RSP: 0018:ffff880703603d10 EFLAGS: 00000292
[ 8673.343773] RAX: 0000000000000000 RBX: ffff880703603d98 RCX: 0000000000000000
[ 8673.352763] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8673.361735] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8673.370691] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 8673.379650] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[ 8673.388597] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8673.398617] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8673.406019] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8673.414981] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8673.423937] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8673.432884] Stack:
[ 8673.436090] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 8673.445364] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 8673.454641] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 8673.463952] Call Trace:
[ 8673.467671] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8673.477020] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8673.486189] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8673.495059] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8673.503730] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8673.510927] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8673.520270] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8673.526704] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8673.534975] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8673.541982] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8673.550240] Code: 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f 5d <c3> 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d 88
[ 8701.145595] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8701.155222] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8701.286903] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8701.300188] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8701.316428] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8701.325810] RIP: 0010:[<ffffffffa121d1fc>] [<ffffffffa121d1fc>] osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[ 8701.338043] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[ 8701.344982] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8701.353960] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8701.362926] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8701.371881] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8701.380823] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8701.389773] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8701.399800] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8701.407216] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8701.416194] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8701.425157] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8701.434106] Stack:
[ 8701.437328] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8701.446615] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8701.455930] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8701.465223] Call Trace:
[ 8701.468952] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8701.478127] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8701.487003] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8701.495687] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8701.502898] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8701.512259] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8701.518692] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8701.526975] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8701.533979] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8701.542258] Code: 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 <48> 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2
[ 8729.146563] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8729.156187] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8729.287992] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8729.301303] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8729.317614] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8729.327018] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[ 8729.338675] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[ 8729.345634] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[ 8729.354627] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8729.363604] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 8729.372563] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 8729.381513] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 8729.390458] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8729.400481] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8729.407883] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8729.416847] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8729.425799] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8729.434745] Stack:
[ 8729.437956] ffff880703603d08 000000009f8f2001 ffffea00027e3c80 000000009f8f2001
[ 8729.447233] ffff880703603d57 ffff880703603d57 ffffffffa121b990 0000000000000000
[ 8729.456519] ffff880703603d78 ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10
[ 8729.465812] Call Trace:
[ 8729.469535] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8729.478694] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8729.487848] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8729.496999] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8729.505861] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8729.514510] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8729.521705] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8729.531051] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8729.537474] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8729.545742] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8729.552733] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8729.560992] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[ 8743.671076] INFO: rcu_sched self-detected stall on CPU { 24} (t=1320042 jiffies g=907112 c=907111 q=1404478)
[ 8743.683386] Task dump for CPU 24:
[ 8743.688111] OI_scrub R running task 0 5551 2 0x00000088
[ 8743.697007] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 8743.706310] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 8743.715594] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 8743.724850] Call Trace:
[ 8743.728513] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 8743.736459] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 8743.743391] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 8743.750889] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 8743.758557] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 8743.766788] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 8743.774324] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 8743.782310] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 8743.789407] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 8743.796975] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 8743.804230] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 8743.811089] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 8743.819096] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 8743.826909] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 8743.834316] <EOI> [<ffffffffa121cfa5>] ? osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[ 8743.844118] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8743.853210] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8743.862107] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8743.870713] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8743.879108] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8743.886050] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8743.895132] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8743.901308] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8743.909326] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8743.916071] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8769.147963] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8769.157338] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8769.286866] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8769.300048] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8769.316119] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8769.325423] RIP: 0010:[<ffffffffa121cfb9>] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 8769.337564] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[ 8769.344429] RAX: 00000000000072ce RBX: ffff880703603d57 RCX: ffff88042bd78800
[ 8769.353339] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 8769.362234] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8769.371131] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8769.380037] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[ 8769.388934] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8769.398897] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8769.406245] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8769.415147] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8769.424053] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8769.432948] Stack:
[ 8769.436113] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8769.445348] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8769.454602] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8769.463854] Call Trace:
[ 8769.467528] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8769.476663] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8769.485505] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8769.494149] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8769.501340] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8769.510680] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8769.517078] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8769.525321] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8769.532301] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8769.540547] Code: 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff <42> 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83
[ 8797.148932] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8797.158526] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8797.290259] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8797.303571] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8797.319869] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8797.329272] RIP: 0010:[<ffffffffa121b995>] [<ffffffffa121b995>] osd_scrub_exec+0x5/0x5b0 [osd_ldiskfs]
[ 8797.340838] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[ 8797.347796] RAX: ffffffffa121b990 RBX: ffff880703603d98 RCX: 0000000000000000
[ 8797.356777] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8797.365740] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8797.374701] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 8797.383645] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[ 8797.392581] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8797.402603] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8797.410000] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8797.418953] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8797.427902] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8797.436845] Stack:
[ 8797.440047] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 8797.449306] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 8797.458587] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 8797.467876] Call Trace:
[ 8797.471589] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8797.480943] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8797.490106] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8797.498962] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8797.507620] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8797.514820] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8797.524158] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8797.530579] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8797.538854] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8797.545862] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8797.554121] Code: 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 <55> 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83
[ 8825.149909] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8825.159522] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8825.291092] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8825.304374] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8825.320626] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8825.330005] RIP: 0010:[<ffffffffa1217b89>] [<ffffffffa1217b89>] osd_scrub_next+0x169/0x3c0 [osd_ldiskfs]
[ 8825.341741] RSP: 0018:ffff880703603d08 EFLAGS: 00000282
[ 8825.348674] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[ 8825.357646] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 8825.366605] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[ 8825.375549] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8825.384488] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000202
[ 8825.393426] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8825.403438] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8825.410831] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8825.419789] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8825.428737] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8825.437674] Stack:
[ 8825.440875] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[ 8825.450176] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 8825.459471] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 8825.468742] Call Trace:
[ 8825.472454] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8825.481620] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8825.490767] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8825.499624] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8825.508281] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8825.515478] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8825.524818] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8825.531236] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8825.539500] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8825.546503] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8825.554766] Code: df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 48 8b 05 be 58 9c ff 48 85 c0 0f 84 c2 fe ff ff 0f 1f 44
[ 8853.150884] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8853.160498] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8853.292053] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8853.305334] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8853.321569] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8853.330944] RIP: 0010:[<ffffffffa121cfb9>] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 8853.343160] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[ 8853.350092] RAX: 00000000000072ce RBX: ffff880703603d57 RCX: ffff88042bd78800
[ 8853.359062] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 8853.368020] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8853.376959] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8853.385902] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[ 8853.394841] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8853.404861] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8853.412261] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8853.421223] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8853.430177] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8853.439119] Stack:
[ 8853.442327] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8853.451607] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8853.460906] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8853.470218] Call Trace:
[ 8853.473937] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8853.483101] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8853.491973] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8853.500639] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8853.507844] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8853.517186] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8853.523616] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8853.531897] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8853.538916] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8853.547193] Code: 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff <42> 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83
[ 8881.151858] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8881.161505] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8881.293327] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8881.306638] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8881.322938] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8881.332345] RIP: 0010:[<ffffffffa121d1fc>] [<ffffffffa121d1fc>] osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[ 8881.344587] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[ 8881.351549] RAX: 0000000020000000 RBX: ffff880703603d98 RCX: 0000000000000000
[ 8881.360529] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8881.369506] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8881.378470] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 8881.387420] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[ 8881.396366] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8881.406391] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8881.413794] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8881.422757] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8881.431715] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8881.440657] Stack:
[ 8881.443862] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8881.453137] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8881.462426] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8881.471719] Call Trace:
[ 8881.475436] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8881.484612] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8881.493482] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8881.502152] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8881.509366] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8881.518713] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8881.525141] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8881.533412] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8881.540419] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8881.548686] Code: 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 <48> 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2
[ 8909.152833] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 8909.162446] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8909.294151] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8909.307457] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8909.323746] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8909.333143] RIP: 0010:[<ffffffffa121d1ca>] [<ffffffffa121d1ca>] osd_inode_iteration+0x47a/0xcc0 [osd_ldiskfs]
[ 8909.345374] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[ 8909.352330] RAX: 0000000000000004 RBX: ffff880703603d57 RCX: ffff880703603d98
[ 8909.361312] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 8909.370286] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8909.379245] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8909.388192] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[ 8909.397138] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8909.407158] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8909.414560] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8909.423523] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8909.432485] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8909.441434] Stack:
[ 8909.444648] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8909.453938] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8909.463221] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8909.472514] Call Trace:
[ 8909.476245] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8909.485418] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8909.494284] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8909.502971] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8909.510207] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8909.519553] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8909.525991] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8909.534274] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8909.541294] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8909.549567] Code: 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff <48> 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[ 8923.680346] INFO: rcu_sched self-detected stall on CPU { 24} (t=1500045 jiffies g=907112 c=907111 q=1415359)
[ 8923.692648] Task dump for CPU 24:
[ 8923.697398] OI_scrub R running task 0 5551 2 0x00000088
[ 8923.706319] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 8923.715643] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 8923.724960] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 8923.734272] Call Trace:
[ 8923.737975] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 8923.745963] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 8923.752940] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 8923.760477] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 8923.768185] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 8923.776457] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 8923.784027] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 8923.792069] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 8923.799212] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 8923.806816] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 8923.814118] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 8923.821000] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 8923.829040] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 8923.836871] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 8923.844289] <EOI> [<ffffffffa121d1e0>] ? osd_inode_iteration+0x490/0xcc0 [osd_ldiskfs]
[ 8923.854091] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 8923.863197] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8923.872109] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8923.880732] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8923.889144] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8923.896102] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8923.905182] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8923.911349] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8923.919370] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8923.926118] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8949.154233] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8949.163628] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8949.293143] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8949.306317] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8949.322378] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8949.331682] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[ 8949.343237] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[ 8949.350095] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[ 8949.359001] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8949.367898] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 8949.376794] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 8949.385691] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 8949.394583] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8949.404550] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8949.411894] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8949.420801] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8949.429708] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8949.438606] Stack:
[ 8949.441774] ffff880703603d08 0000000000000001 ffff880703603df0 000000009f8f2001
[ 8949.451011] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[ 8949.460258] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[ 8949.469507] Call Trace:
[ 8949.473189] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 8949.482324] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8949.491454] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8949.500296] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8949.508951] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8949.516137] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8949.525455] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8949.531851] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8949.540091] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8949.547072] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8949.555317] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[ 8977.155202] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 8977.164799] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 8977.296298] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 8977.309581] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 8977.325830] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 8977.335211] RIP: 0010:[<ffffffffa121d1d9>] [<ffffffffa121d1d9>] osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[ 8977.347427] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[ 8977.354366] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 8977.363341] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 8977.372312] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 8977.381263] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 8977.390207] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 8977.399153] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 8977.409170] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8977.416573] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 8977.425536] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8977.434487] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 8977.443424] Stack:
[ 8977.446627] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 8977.455903] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 8977.465192] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 8977.474469] Call Trace:
[ 8977.478181] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 8977.487345] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 8977.496223] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 8977.504891] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 8977.512098] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 8977.521442] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 8977.527874] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8977.536151] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 8977.543165] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 8977.551438] Code: 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5
[ 9005.156184] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9005.165817] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9005.297689] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9005.310999] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9005.327290] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9005.336695] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[ 9005.348944] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[ 9005.355915] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[ 9005.364912] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 9005.373892] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9005.382860] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9005.391815] R13: ffff880703603df0 R14: 000000009f8f2001 R15: ffff880703603d08
[ 9005.400761] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9005.410799] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9005.418211] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9005.427183] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9005.436150] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9005.445103] Stack:
[ 9005.448322] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9005.457616] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9005.466906] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9005.476214] Call Trace:
[ 9005.479933] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9005.489096] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9005.497975] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9005.506655] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9005.513876] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9005.523233] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9005.529676] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9005.537957] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9005.544974] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9005.553252] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[ 9033.157155] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9033.166791] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9033.298619] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9033.311935] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9033.328240] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9033.337655] RIP: 0010:[<ffffffffa121cffc>] [<ffffffffa121cffc>] osd_inode_iteration+0x2ac/0xcc0 [osd_ldiskfs]
[ 9033.349907] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[ 9033.356876] RAX: 0000000000000004 RBX: ffff880703603d98 RCX: ffff880703603d98
[ 9033.365873] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 9033.374863] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 9033.383834] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 9033.392792] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[ 9033.401742] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9033.411766] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9033.419169] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9033.428130] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9033.437085] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9033.446026] Stack:
[ 9033.449230] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9033.458515] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9033.467792] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9033.477095] Call Trace:
[ 9033.480809] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9033.489966] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9033.498821] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9033.507486] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9033.514694] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9033.524036] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9033.530458] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9033.538727] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9033.545732] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9033.553995] Code: 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 <41> 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 0f
[ 9061.158128] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9061.167747] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9061.299416] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9061.312725] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9061.329001] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9061.338398] RIP: 0010:[<ffffffffa121ba25>] [<ffffffffa121ba25>] osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[ 9061.350044] RSP: 0018:ffff880703603d08 EFLAGS: 00000292
[ 9061.356996] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 9061.365981] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9061.374952] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9061.383907] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9061.392848] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000286
[ 9061.401783] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9061.411793] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9061.419184] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9061.428134] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9061.437085] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9061.446026] Stack:
[ 9061.449231] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[ 9061.458512] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 9061.467778] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 9061.477063] Call Trace:
[ 9061.480781] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9061.489942] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9061.499100] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9061.507967] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9061.516635] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9061.523841] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9061.533184] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9061.539612] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9061.547888] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9061.554897] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9061.563166] Code: 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d
[ 9089.159104] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9089.168717] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9089.300264] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9089.313535] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9089.329767] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9089.339145] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[ 9089.350765] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[ 9089.357695] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9089.366664] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9089.375626] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9089.384568] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9089.393509] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9089.402449] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9089.412467] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9089.419865] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9089.428825] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9089.437779] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9089.446719] Stack:
[ 9089.449925] ffff880703603d08 ffff880703603d57 ffffea00027e3c80 000000009f8f2001
[ 9089.459209] ffff880703603d57 ffff880703603d57 0000000000000000 ffff88042bd78800
[ 9089.468501] 00000000000072ce ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[ 9089.477793] Call Trace:
[ 9089.481506] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9089.490668] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9089.499829] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9089.508684] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9089.517343] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9089.524544] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9089.533889] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9089.540314] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9089.548585] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9089.555589] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9089.563852] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[ 9103.689617] INFO: rcu_sched self-detected stall on CPU { 24} (t=1680048 jiffies g=907112 c=907111 q=1422050)
[ 9103.701962] Task dump for CPU 24:
[ 9103.706694] OI_scrub R running task 0 5551 2 0x00000088
[ 9103.715606] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 9103.724918] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 9103.734203] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 9103.743478] Call Trace:
[ 9103.747162] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 9103.755119] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 9103.762075] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 9103.769590] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 9103.777270] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 9103.785512] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 9103.793058] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 9103.801067] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 9103.808178] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 9103.815762] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 9103.823028] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 9103.829885] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 9103.837904] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 9103.845718] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 9103.853144] <EOI> [<ffffffffa121cfd2>] ? osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[ 9103.862950] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 9103.872047] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9103.880953] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9103.889559] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9103.897960] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9103.904904] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9103.913982] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9103.920155] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9103.928174] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9103.934933] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9129.160503] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9129.169899] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9129.299494] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9129.312676] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9129.328757] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9129.338059] RIP: 0010:[<ffffffffa121b990>] [<ffffffffa121b990>] osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9129.350006] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[ 9129.356868] RAX: ffffffffa121b990 RBX: 0000000000000000 RCX: 0000000000000000
[ 9129.365772] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9129.374670] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9129.383576] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 9129.392481] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[ 9129.401375] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9129.411331] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9129.418672] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9129.427582] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9129.436483] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9129.445377] Stack:
[ 9129.448541] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 9129.457783] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 9129.467031] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 9129.476287] Call Trace:
[ 9129.479978] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9129.489300] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9129.498436] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9129.507281] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9129.515927] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9129.523115] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9129.532435] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9129.538834] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9129.547078] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9129.554067] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9129.562309] Code: b5 07 00 00 c7 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45
[ 9157.161480] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9157.171073] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9157.302756] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9157.316045] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9157.332314] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9157.341700] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[ 9157.353344] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[ 9157.360286] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9157.369265] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9157.378230] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9157.387190] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9157.396140] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9157.405090] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9157.415120] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9157.422527] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9157.431497] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9157.440456] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9157.449405] Stack:
[ 9157.452620] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 ffff880703603d57
[ 9157.461909] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 9157.471190] ffffffffa121b990 0000000000000000 ffff880703603d78 ffff8800b2a36000
[ 9157.480472] Call Trace:
[ 9157.484196] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9157.493353] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9157.502510] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9157.511657] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9157.520507] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9157.529164] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9157.536360] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9157.545694] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9157.552118] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9157.560389] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9157.567387] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9157.575639] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[ 9185.162448] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9185.172054] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9185.303362] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9185.316621] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9185.332812] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9185.342169] RIP: 0010:[<ffffffffa121d1e9>] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9185.354358] RSP: 0018:ffff880703603d18 EFLAGS: 00000292
[ 9185.361282] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[ 9185.370250] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9185.379198] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9185.388138] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 9185.397080] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[ 9185.406023] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9185.416036] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9185.423428] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9185.432373] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9185.441311] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9185.450236] Stack:
[ 9185.453434] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9185.462722] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9185.471984] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9185.481261] Call Trace:
[ 9185.484968] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9185.494126] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9185.502983] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9185.511635] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9185.518839] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9185.528182] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9185.534618] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9185.542892] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9185.549903] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9185.558180] Code: 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 <85> c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 48 8b
[ 9213.163423] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9213.173046] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9213.304823] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9213.318127] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9213.334409] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9213.343816] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[ 9213.356057] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[ 9213.363016] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 9213.372003] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 9213.380981] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9213.389943] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9213.398894] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 9213.407842] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9213.417863] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9213.425266] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9213.434228] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9213.443188] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9213.452136] Stack:
[ 9213.455346] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9213.464628] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9213.473908] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9213.483205] Call Trace:
[ 9213.486925] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9213.496089] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9213.504959] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9213.513632] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9213.520835] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9213.530177] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9213.536606] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9213.544879] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9213.551896] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9213.560174] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[ 9241.164399] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9241.174039] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9241.305878] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9241.319183] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9241.335470] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9241.344882] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[ 9241.356549] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 9241.363517] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9241.372502] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9241.381478] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9241.390444] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9241.399393] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9241.408338] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9241.418362] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9241.425772] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9241.434736] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9241.443696] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9241.452646] Stack:
[ 9241.455855] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 0000000000000001
[ 9241.465134] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 9241.474404] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[ 9241.483700] Call Trace:
[ 9241.487407] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9241.496569] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9241.505726] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9241.514590] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9241.523258] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9241.530459] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9241.539788] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9241.546217] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9241.554488] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9241.561505] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9241.569766] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[ 9269.165380] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9269.175004] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9269.306578] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9269.319860] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9269.336107] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9269.345492] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[ 9269.357121] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[ 9269.364056] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9269.373029] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9269.381991] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9269.390940] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9269.399886] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9269.408829] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9269.418855] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9269.426258] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9269.435212] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9269.444164] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9269.453102] Stack:
[ 9269.456298] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[ 9269.465577] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 9269.474862] 0000000020000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[ 9269.484141] Call Trace:
[ 9269.487854] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9269.497015] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9269.506173] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9269.515041] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9269.523702] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9269.530909] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9269.540257] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9269.546683] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9269.554956] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9269.561971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9269.570244] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[ 9283.698887] INFO: rcu_sched self-detected stall on CPU { 24} (t=1860051 jiffies g=907112 c=907111 q=1432823)
[ 9283.711229] Task dump for CPU 24:
[ 9283.715957] OI_scrub R running task 0 5551 2 0x00000088
[ 9283.724867] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 9283.734190] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 9283.743475] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 9283.752736] Call Trace:
[ 9283.756414] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 9283.764381] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 9283.771332] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 9283.778840] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 9283.786527] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 9283.794772] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 9283.802316] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 9283.810325] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 9283.817438] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 9283.825023] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 9283.832291] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 9283.839162] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 9283.847189] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 9283.855005] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 9283.862441] <EOI> [<ffffffffa1217b81>] ? osd_scrub_next+0x161/0x3c0 [osd_ldiskfs]
[ 9283.871756] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 9283.880655] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9283.889550] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9283.898161] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9283.906559] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9283.913507] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9283.922585] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9283.928750] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9283.936761] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9283.943514] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9309.166768] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9309.176157] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9309.305604] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9309.318778] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9309.334840] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9309.344137] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[ 9309.356277] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[ 9309.363136] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 9309.372040] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9309.380937] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9309.389833] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9309.398728] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffff880703603d08
[ 9309.407623] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9309.417598] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9309.424941] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9309.433839] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9309.442745] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9309.451645] Stack:
[ 9309.454813] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9309.464061] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9309.473317] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9309.482573] Call Trace:
[ 9309.486271] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9309.495404] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9309.504247] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9309.512891] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9309.520082] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9309.529408] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9309.535808] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9309.544048] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9309.551034] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9309.559286] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[ 9337.167749] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9337.177347] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9337.309158] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9337.322466] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9337.338766] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9337.348171] RIP: 0010:[<ffffffffa1162b30>] [<ffffffffa1162b30>] ldiskfs_used_dirs_count+0x30/0x30 [ldiskfs]
[ 9337.360244] RSP: 0018:ffff880703603d10 EFLAGS: 00000286
[ 9337.367215] RAX: ffff8800b2a37468 RBX: 0000000000000000 RCX: 0000000000000000
[ 9337.376209] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 9337.385190] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9337.394168] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 9337.403122] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[ 9337.412070] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9337.422093] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9337.429496] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9337.438459] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9337.447423] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9337.456364] Stack:
[ 9337.459570] ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 9337.468851] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 9337.478136] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 9337.487416] Call Trace:
[ 9337.491143] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 9337.500500] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9337.509664] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9337.518536] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9337.527200] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9337.534398] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9337.543743] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9337.550179] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9337.558454] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9337.565469] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9337.573742] Code: 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 <66> 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89
[ 9365.168719] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9365.178354] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9365.310090] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9365.323382] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9365.339638] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9365.349026] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[ 9365.361258] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[ 9365.368212] RAX: ffff88042bd78800 RBX: ffff880703603d98 RCX: ffff88042bd78800
[ 9365.377196] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 9365.386165] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9365.395131] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 9365.404091] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[ 9365.413040] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9365.423064] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9365.430473] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9365.439434] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9365.448389] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9365.457336] Stack:
[ 9365.460543] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9365.469835] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9365.479124] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9365.488406] Call Trace:
[ 9365.492122] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9365.501284] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9365.510153] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9365.518822] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9365.526025] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9365.535364] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9365.541795] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9365.550074] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9365.557087] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9365.565361] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[ 9393.169701] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9393.179310] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9393.311038] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9393.324340] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9393.340623] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9393.350019] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[ 9393.362258] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[ 9393.369215] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 9393.378200] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9393.387173] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9393.396133] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9393.405086] R13: ffff880703603df0 R14: 000000009f8f2001 R15: ffff880703603d08
[ 9393.414031] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9393.424056] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9393.431459] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9393.440422] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9393.449377] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9393.458329] Stack:
[ 9393.461544] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9393.470839] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9393.480126] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9393.489420] Call Trace:
[ 9393.493146] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9393.502317] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9393.511195] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9393.519874] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9393.527093] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9393.536441] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9393.542875] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9393.551145] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9393.558168] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9393.566448] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[ 9421.170669] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9421.180298] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9421.312162] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9421.325472] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9421.341791] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9421.351197] RIP: 0010:[<ffffffffa121b9a2>] [<ffffffffa121b9a2>] osd_scrub_exec+0x12/0x5b0 [osd_ldiskfs]
[ 9421.362860] RSP: 0018:ffff880703603cf0 EFLAGS: 00000293
[ 9421.369822] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 9421.378802] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9421.387788] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9421.396756] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c70
[ 9421.405712] R13: 0000000000000202 R14: 0000000000000010 R15: ffffffffa1217b69
[ 9421.414652] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9421.424673] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9421.432079] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9421.441043] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9421.450000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9421.458942] Stack:
[ 9421.462149] 0000000000000004 00000000ffffffff 0000000000000001 ffff880703603df0
[ 9421.471430] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 9421.480730] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 9421.490031] Call Trace:
[ 9421.493759] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9421.502940] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9421.512100] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9421.520966] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9421.529627] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9421.536838] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9421.546186] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9421.552618] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9421.560880] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9421.567887] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9421.576156] Code: 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <41> 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90
[ 9449.171645] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9449.181279] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9449.312901] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9449.326186] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9449.342430] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9449.351819] RIP: 0010:[<ffffffffa121d1d9>] [<ffffffffa121d1d9>] osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[ 9449.364051] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[ 9449.370990] RAX: 0000000000000004 RBX: ffff880703603d98 RCX: 0000000000000000
[ 9449.379966] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9449.388933] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9449.397884] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 9449.406832] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[ 9449.415780] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9449.425810] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9449.433209] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9449.442171] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9449.451123] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9449.460064] Stack:
[ 9449.463270] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9449.472566] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9449.481857] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9449.491143] Call Trace:
[ 9449.494856] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9449.504016] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9449.512882] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9449.521548] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9449.528758] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9449.538098] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9449.544532] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9449.552811] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9449.559820] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9449.568088] Code: 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5
[ 9463.708158] INFO: rcu_sched self-detected stall on CPU { 24} (t=2040054 jiffies g=907112 c=907111 q=1439501)
[ 9463.720487] Task dump for CPU 24:
[ 9463.725229] OI_scrub R running task 0 5551 2 0x00000088
[ 9463.734143] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 9463.743467] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 9463.752764] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 9463.762057] Call Trace:
[ 9463.765753] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 9463.773734] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 9463.780701] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 9463.788230] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 9463.795930] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 9463.804195] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 9463.811765] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 9463.819792] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 9463.826928] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 9463.834529] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 9463.841813] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 9463.848687] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 9463.856718] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 9463.864541] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 9463.871964] <EOI> [<ffffffffa1217b87>] ? osd_scrub_next+0x167/0x3c0 [osd_ldiskfs]
[ 9463.881276] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 9463.890171] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9463.899063] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9463.907665] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9463.916073] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9463.923023] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9463.932105] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9463.938274] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9463.946283] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9463.953039] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9489.173038] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9489.182437] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9489.311963] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9489.325131] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9489.341186] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9489.350478] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[ 9489.362023] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[ 9489.368876] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9489.377771] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9489.386661] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9489.395551] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9489.404441] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9489.413328] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9489.423288] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9489.430624] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9489.439523] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9489.448421] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9489.457312] Stack:
[ 9489.460473] ffff880703603d08 000000009f8f2001 ffff880703603df0 000000009f8f2001
[ 9489.469714] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 9489.478947] 0000000000000004 0000000000000000 ffff880703603d78 ffff8800b2a36000
[ 9489.488192] Call Trace:
[ 9489.491874] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9489.501001] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9489.510124] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9489.518957] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9489.527598] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9489.534770] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9489.544089] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9489.550484] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9489.558730] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9489.565719] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9489.573952] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[ 9517.174013] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9517.183614] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9517.315222] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9517.328511] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9517.344762] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9517.354144] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[ 9517.365788] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 9517.372722] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9517.381694] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9517.390653] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9517.399602] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9517.408545] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9517.417490] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9517.427514] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9517.434918] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9517.443871] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9517.452821] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9517.461759] Stack:
[ 9517.464953] ffff880703603d08 0000000000000001 ffffea00027e3c80 000000009f8f2001
[ 9517.474241] ffff880703603d57 0000000000000001 ffffffffa1217a20 ffff880703603d98
[ 9517.483533] ffff880703603d78 ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10
[ 9517.492818] Call Trace:
[ 9517.496526] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9517.505392] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9517.514549] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9517.523687] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9517.532546] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9517.541208] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9517.548415] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9517.557753] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9517.564184] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9517.572459] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9517.579466] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9517.587736] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[ 9545.174996] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9545.184603] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9545.315987] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9545.329249] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9545.345457] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9545.354820] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[ 9545.366439] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 9545.373365] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9545.382323] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9545.391274] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9545.400224] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9545.409168] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9545.418122] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9545.428150] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9545.435550] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9545.444502] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9545.453448] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9545.462381] Stack:
[ 9545.465585] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[ 9545.474866] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 9545.484154] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff880419dc3a00
[ 9545.493428] Call Trace:
[ 9545.497131] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9545.506291] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9545.515440] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9545.524284] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9545.532937] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9545.540140] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9545.549480] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9545.555908] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9545.564183] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9545.571187] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9545.579450] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[ 9573.175964] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9573.185585] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9573.317166] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9573.330449] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9573.346686] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9573.356067] RIP: 0010:[<ffffffffa121cfda>] [<ffffffffa121cfda>] osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[ 9573.368292] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[ 9573.375230] RAX: ffff88042bd78800 RBX: ffff880703603d98 RCX: ffff88042bd78800
[ 9573.384206] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 9573.393165] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 9573.402117] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 9573.411065] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[ 9573.420012] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9573.430025] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9573.437416] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9573.446374] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9573.455324] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9573.464259] Stack:
[ 9573.467462] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9573.476746] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9573.486030] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9573.495318] Call Trace:
[ 9573.499039] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9573.508199] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9573.517067] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9573.525731] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9573.532935] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9573.542285] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9573.548706] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9573.556986] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9573.564003] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9573.572271] Code: a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff <48> 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[ 9601.176946] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9601.186575] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9601.318391] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9601.331703] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9601.348002] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9601.357405] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[ 9601.369070] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[ 9601.376027] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[ 9601.385013] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9601.393998] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[ 9601.402967] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c70
[ 9601.411920] R13: 0000000000000282 R14: 0000000000000010 R15: ffffffffa1217a4b
[ 9601.420869] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9601.430893] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9601.438291] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9601.447249] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9601.456206] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9601.465151] Stack:
[ 9601.468361] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[ 9601.477648] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 9601.486929] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 9601.496227] Call Trace:
[ 9601.499951] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 9601.509126] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9601.518290] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9601.527162] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9601.535828] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9601.543034] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9601.552377] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9601.558812] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9601.567094] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9601.574102] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9601.582377] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[ 9629.177917] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9629.187543] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9629.319182] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9629.332470] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9629.348727] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9629.358119] RIP: 0010:[<ffffffffa121ba09>] [<ffffffffa121ba09>] osd_scrub_exec+0x79/0x5b0 [osd_ldiskfs]
[ 9629.369762] RSP: 0018:ffff880703603c78 EFLAGS: 00000202
[ 9629.376701] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 00000000fb6db880
[ 9629.385677] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9629.394643] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9629.403594] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9629.412544] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9629.421496] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9629.431525] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9629.438933] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9629.447900] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9629.456858] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9629.465805] Stack:
[ 9629.469017] ffff880703603d08 0000000000000001 ffffea00027e3c80 000000009f8f2001
[ 9629.478305] ffff880703603d57 ffff880703603d57 0000000000000000 ffff88042bd78800
[ 9629.487597] 00000000000072ce ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[ 9629.496903] Call Trace:
[ 9629.500624] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9629.509792] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9629.518953] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9629.527818] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9629.536485] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9629.543699] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9629.553059] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9629.559494] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9629.567778] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9629.574789] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9629.583065] Code: 07 74 4f 41 83 f9 02 74 78 48 89 ca 44 89 c9 e8 0e c6 ff ff 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 <65> 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41
[ 9643.717428] INFO: rcu_sched self-detected stall on CPU { 24} (t=2220057 jiffies g=907112 c=907111 q=1450326)
[ 9643.729781] Task dump for CPU 24:
[ 9643.734514] OI_scrub R running task 0 5551 2 0x00000088
[ 9643.743428] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 9643.752747] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 9643.762046] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 9643.771332] Call Trace:
[ 9643.775013] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 9643.782976] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 9643.789931] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 9643.797436] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 9643.805121] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 9643.813388] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 9643.820937] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 9643.828953] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 9643.836078] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 9643.843668] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 9643.850942] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 9643.857830] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 9643.865855] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 9643.873669] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 9643.881101] <EOI> [<ffffffffa121d1ee>] ? osd_inode_iteration+0x49e/0xcc0 [osd_ldiskfs]
[ 9643.890909] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9643.900018] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9643.908927] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9643.917555] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9643.925974] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9643.932932] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9643.942013] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9643.948183] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9643.956211] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9643.962966] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9669.179310] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9669.188695] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9669.318246] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9669.331422] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9669.347486] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9669.356784] RIP: 0010:[<ffffffffa121cffc>] [<ffffffffa121cffc>] osd_inode_iteration+0x2ac/0xcc0 [osd_ldiskfs]
[ 9669.368926] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[ 9669.375779] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[ 9669.384687] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 9669.393587] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 9669.402482] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9669.411385] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 9669.420284] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9669.430251] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9669.437594] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9669.446498] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9669.455402] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9669.464297] Stack:
[ 9669.467464] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9669.476713] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9669.485962] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9669.495219] Call Trace:
[ 9669.498901] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9669.508042] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9669.516875] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9669.525522] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9669.532719] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9669.542038] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9669.548435] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9669.556690] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9669.563660] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9669.571906] Code: 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 <41> 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 0f
[ 9697.180284] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9697.189868] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9697.321625] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9697.334929] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9697.351224] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9697.360625] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[ 9697.372284] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[ 9697.379243] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9697.388227] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9697.397202] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9697.406162] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9697.415113] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9697.424059] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9697.434085] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9697.441494] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9697.450459] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9697.459437] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9697.468386] Stack:
[ 9697.471600] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 0000000000000001
[ 9697.480891] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[ 9697.490184] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[ 9697.499483] Call Trace:
[ 9697.503211] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9697.512383] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9697.521544] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9697.530414] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9697.539082] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9697.546292] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9697.555638] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9697.562064] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9697.570334] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9697.577338] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9697.585609] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[ 9725.181259] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9725.190894] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9725.322493] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9725.335777] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9725.352022] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9725.361402] RIP: 0010:[<ffffffffa1162b55>] [<ffffffffa1162b55>] ldiskfs_itable_unused_count+0x25/0x30 [ldiskfs]
[ 9725.373830] RSP: 0018:ffff880703603d08 EFLAGS: 00000202
[ 9725.380768] RAX: 00000000000072ce RBX: 0000000000000001 RCX: ffff88042bd78800
[ 9725.389750] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 9725.398706] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9725.407653] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9725.416596] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[ 9725.425536] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9725.435550] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9725.442943] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9725.451894] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9725.460835] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9725.469765] Stack:
[ 9725.472958] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[ 9725.482231] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[ 9725.491522] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[ 9725.500803] Call Trace:
[ 9725.504519] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 9725.513664] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9725.522815] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9725.531670] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9725.540330] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9725.547541] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9725.556875] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9725.563295] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9725.571566] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9725.578566] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9725.586824] Code: 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76 07 0f b7 46 32 c1 e0 10 09 d0 <5d> c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 89 16 48 8b
[ 9753.182241] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9753.191850] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9753.323400] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9753.336679] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9753.352917] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9753.362294] RIP: 0010:[<ffffffffa1217a20>] [<ffffffffa1217a20>] osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9753.374026] RSP: 0018:ffff880703603d10 EFLAGS: 00000287
[ 9753.380959] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[ 9753.389927] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9753.398881] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 9753.407823] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9753.416763] R13: ffff880703603df0 R14: 000000009f8f2001 R15: ffff880703603d08
[ 9753.425710] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9753.435732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9753.443125] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9753.452089] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9753.461037] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9753.469981] Stack:
[ 9753.473191] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 9753.482503] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 9753.491798] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 9753.501094] Call Trace:
[ 9753.504814] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 9753.514176] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9753.523366] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9753.532236] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9753.540901] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9753.548106] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9753.557457] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9753.563901] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9753.572183] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9753.579206] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9753.587487] Code: f7 48 8d 04 80 48 c1 e0 03 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd
[ 9781.183216] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9781.192849] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9781.324586] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9781.337881] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9781.354151] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9781.363542] RIP: 0010:[<ffffffffa121ba26>] [<ffffffffa121ba26>] osd_scrub_exec+0x96/0x5b0 [osd_ldiskfs]
[ 9781.375200] RSP: 0018:ffff880703603d10 EFLAGS: 00000292
[ 9781.382146] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 9781.391129] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9781.400109] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9781.409069] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9781.418023] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 9781.426979] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9781.437014] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9781.444426] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9781.453394] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9781.462356] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9781.471302] Stack:
[ 9781.474515] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[ 9781.483800] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[ 9781.493102] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[ 9781.502395] Call Trace:
[ 9781.506116] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9781.515479] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9781.524648] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9781.533518] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9781.542188] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9781.549399] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9781.558747] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9781.565180] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9781.573460] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9781.580473] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9781.588751] Code: 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f 5d <c3> 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d 88
[ 9809.184192] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9809.193813] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9809.325456] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9809.338745] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9809.354999] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9809.364387] RIP: 0010:[<ffffffffa121cfb9>] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 9809.376612] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[ 9809.383556] RAX: 00000000000072ce RBX: ffff880703603d98 RCX: ffff88042bd78800
[ 9809.392539] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 9809.401503] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9809.410460] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[ 9809.419408] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[ 9809.428358] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9809.438382] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9809.445784] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9809.454745] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9809.463697] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9809.472637] Stack:
[ 9809.475840] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9809.485129] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9809.494428] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9809.503712] Call Trace:
[ 9809.507423] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9809.516583] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9809.525448] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9809.534106] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9809.541313] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9809.550654] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9809.557076] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9809.565344] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9809.572353] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9809.580621] Code: 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff <42> 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83
[ 9823.726699] INFO: rcu_sched self-detected stall on CPU { 24} (t=2400060 jiffies g=907112 c=907111 q=1457053)
[ 9823.739027] Task dump for CPU 24:
[ 9823.743757] OI_scrub R running task 0 5551 2 0x00000088
[ 9823.752674] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[ 9823.761990] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[ 9823.771306] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[ 9823.780601] Call Trace:
[ 9823.784282] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[ 9823.792254] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[ 9823.799216] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[ 9823.806752] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[ 9823.814439] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[ 9823.822707] [<ffffffff81099697>] update_process_times+0x47/0x80
[ 9823.830272] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[ 9823.838298] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[ 9823.845440] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[ 9823.853032] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[ 9823.860305] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[ 9823.867186] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[ 9823.875219] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[ 9823.883040] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[ 9823.890463] <EOI> [<ffffffffa121cfda>] ? osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[ 9823.900255] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[ 9823.909350] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9823.918249] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9823.926866] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9823.935278] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9823.942238] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9823.951329] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9823.957505] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9823.965520] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9823.972287] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9849.185579] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9849.194976] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9849.324562] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9849.337728] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9849.353804] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9849.363106] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[ 9849.375249] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[ 9849.382103] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[ 9849.391013] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[ 9849.399913] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9849.408822] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[ 9849.417725] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 9849.426613] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9849.436581] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9849.443928] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9849.452838] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9849.461747] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9849.470649] Stack:
[ 9849.473821] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9849.483072] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9849.492318] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9849.501565] Call Trace:
[ 9849.505250] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9849.514390] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9849.523238] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9849.531887] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9849.539081] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9849.548415] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9849.554825] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9849.563080] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9849.570070] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9849.578316] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[ 9877.186554] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9877.196155] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9877.328006] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9877.341327] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9877.357624] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9877.367039] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[ 9877.379281] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[ 9877.386247] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 9877.395239] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9877.404223] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9877.413192] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 9877.422151] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 9877.431107] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9877.441139] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9877.448548] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9877.457519] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9877.466486] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9877.475437] Stack:
[ 9877.478655] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9877.487951] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9877.497234] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9877.506545] Call Trace:
[ 9877.510271] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9877.519442] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9877.528320] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9877.537000] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9877.544218] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9877.553557] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9877.559991] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9877.568277] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9877.575300] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9877.583582] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[ 9905.187530] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[ 9905.197161] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9905.328966] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9905.342276] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9905.358580] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9905.367987] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[ 9905.379659] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[ 9905.386613] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[ 9905.395603] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9905.404592] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[ 9905.413564] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9905.422512] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9905.431462] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9905.441498] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9905.448907] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9905.457882] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9905.466845] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9905.475783] Stack:
[ 9905.478986] ffff880703603df0 000000009f8f2001 ffffea00027e3c80 000000009f8f2001
[ 9905.488266] ffff880703603d57 ffff880703603d57 ffff8800b2a37468 0000000000000000
[ 9905.497544] ffff880703603d78 ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[ 9905.506850] Call Trace:
[ 9905.510569] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[ 9905.519732] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9905.528890] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9905.537756] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9905.546416] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9905.553616] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9905.562956] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9905.569378] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9905.577645] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9905.584662] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9905.592938] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[ 9933.188513] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9933.198131] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9933.329673] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9933.342953] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9933.359187] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9933.368564] RIP: 0010:[<ffffffffa121cffc>] [<ffffffffa121cffc>] osd_inode_iteration+0x2ac/0xcc0 [osd_ldiskfs]
[ 9933.380783] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[ 9933.387711] RAX: 0000000000000004 RBX: 0000000000000000 RCX: ffff880703603d98
[ 9933.396675] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[ 9933.405622] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[ 9933.414572] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 9933.423510] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[ 9933.432451] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9933.442466] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9933.449857] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9933.458810] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9933.467766] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9933.476695] Stack:
[ 9933.479880] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9933.489146] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9933.498436] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9933.507707] Call Trace:
[ 9933.511399] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9933.520543] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9933.529407] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9933.538065] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9933.545262] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9933.554598] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9933.561020] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9933.569289] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9933.576294] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9933.584559] Code: 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 <41> 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 0f
[ 9961.189480] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9961.199108] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9961.330870] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9961.344170] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9961.360436] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9961.369833] RIP: 0010:[<ffffffffa1217b69>] [<ffffffffa1217b69>] osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[ 9961.381597] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[ 9961.388550] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[ 9961.397531] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9961.406502] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[ 9961.415458] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[ 9961.424412] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[ 9961.433357] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9961.443381] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9961.450789] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9961.459752] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9961.468712] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9961.477660] Stack:
[ 9961.480874] 000000009f8f2001 ffff880703603d08 ffff880703603d57 ffffea00027e3c80
[ 9961.490166] 000000009f8f2001 ffff880703603d57 ffff880703603d57 0000000000000000
[ 9961.499462] ffff88042bd78800 00000000000072ce ffff880419dc3a00 ffff88042bd79800
[ 9961.508757] Call Trace:
[ 9961.512481] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[ 9961.521644] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9961.530803] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9961.539670] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9961.548322] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9961.555527] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9961.564864] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9961.571292] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9961.579562] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9961.586572] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9961.594839] Code: 45 90 74 49 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 <48> 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83
[ 9989.190462] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[ 9989.200087] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[ 9989.331717] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[ 9989.345006] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[ 9989.361257] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[ 9989.370630] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[ 9989.382859] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[ 9989.389797] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[ 9989.398772] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[ 9989.407736] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[ 9989.416694] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[ 9989.425642] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[ 9989.434605] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[ 9989.444627] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 9989.452024] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[ 9989.460989] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 9989.469943] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 9989.478885] Stack:
[ 9989.482094] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[ 9989.491389] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[ 9989.500686] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[ 9989.509981] Call Trace:
[ 9989.513695] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[ 9989.522865] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[ 9989.531741] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[ 9989.540412] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[ 9989.547624] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[ 9989.556972] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[ 9989.563410] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9989.571697] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[ 9989.578723] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[ 9989.587003] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[10003.735970] INFO: rcu_sched self-detected stall on CPU { 24} (t=2580063 jiffies g=907112 c=907111 q=1468877)
[10003.748283] Task dump for CPU 24:
[10003.753037] OI_scrub R running task 0 5551 2 0x00000088
[10003.761982] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[10003.771305] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[10003.780622] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[10003.789913] Call Trace:
[10003.793615] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[10003.801615] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[10003.808591] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[10003.816135] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[10003.823844] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[10003.832114] [<ffffffff81099697>] update_process_times+0x47/0x80
[10003.839685] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[10003.847727] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[10003.854867] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[10003.862469] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[10003.869766] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[10003.876656] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[10003.884695] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[10003.892525] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[10003.899952] <EOI> [<ffffffffa121b9cd>] ? osd_scrub_exec+0x3d/0x5b0 [osd_ldiskfs]
[10003.909194] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10003.918095] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10003.927010] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10003.935613] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10003.944024] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10003.950982] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10003.960065] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10003.966222] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10003.974229] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10003.980985] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10029.191857] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10029.201226] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10029.331105] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10029.344274] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10029.360332] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10029.369630] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[10029.381768] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[10029.388629] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[10029.397527] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10029.406434] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10029.415339] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10029.424237] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[10029.433137] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10029.443101] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10029.450443] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10029.459340] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10029.468256] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10029.477160] Stack:
[10029.480327] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10029.489577] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10029.498829] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10029.508078] Call Trace:
[10029.511767] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10029.520894] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10029.529737] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10029.538390] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10029.545582] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10029.554912] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10029.561371] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10029.569624] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10029.576605] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10029.584845] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[10057.192832] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10057.202451] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10057.334725] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10057.348033] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10057.364338] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10057.373740] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[10057.386015] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[10057.392987] RAX: ffff8800b2a37468 RBX: ffff880703603d98 RCX: 0000000000000000
[10057.401984] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[10057.410980] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10057.419952] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[10057.428908] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[10057.437865] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10057.447890] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10057.455297] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10057.464262] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10057.473232] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10057.482217] Stack:
[10057.485435] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10057.494720] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10057.504014] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10057.513292] Call Trace:
[10057.517016] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10057.526184] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10057.535059] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10057.543751] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10057.550970] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10057.560322] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10057.566754] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10057.575044] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10057.582059] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10057.590339] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[10085.193806] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10085.203473] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10085.335776] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10085.349077] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10085.365372] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10085.374777] RIP: 0010:[<ffffffffa1217a4b>] [<ffffffffa1217a4b>] osd_scrub_next+0x2b/0x3c0 [osd_ldiskfs]
[10085.386448] RSP: 0018:ffff880703603c70 EFLAGS: 00000282
[10085.393402] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: ffff880703603d98
[10085.402390] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10085.411376] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[10085.420352] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10085.429304] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10085.438254] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10085.448280] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10085.455689] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10085.464665] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10085.473631] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10085.482585] Stack:
[10085.485810] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[10085.495116] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[10085.504409] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[10085.513712] Call Trace:
[10085.517452] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[10085.526619] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10085.535784] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10085.544657] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10085.553330] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10085.560537] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10085.569900] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10085.576322] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10085.584604] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10085.591614] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10085.599880] Code: 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87
[10113.194783] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10113.204427] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10113.336591] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10113.349872] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10113.366120] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10113.375505] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[10113.387743] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[10113.394686] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[10113.403667] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[10113.412629] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10113.421596] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10113.430543] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffff880703603d08
[10113.439498] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10113.449531] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10113.456941] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10113.465912] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10113.474877] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10113.483824] Stack:
[10113.487043] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10113.496338] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10113.505642] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10113.514941] Call Trace:
[10113.518666] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10113.527840] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10113.536723] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10113.545401] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10113.552616] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10113.561991] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10113.568434] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10113.576734] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10113.583757] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10113.592048] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[10141.195759] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10141.205390] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10141.337809] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10141.351142] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10141.367453] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10141.376868] RIP: 0010:[<ffffffffa1217b87>] [<ffffffffa1217b87>] osd_scrub_next+0x167/0x3c0 [osd_ldiskfs]
[10141.388651] RSP: 0018:ffff880703603d00 EFLAGS: 00000282
[10141.395630] RAX: 0000000000000004 RBX: 0000000000000001 RCX: ffff880703603d98
[10141.404626] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[10141.413611] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[10141.422580] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10141.431535] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000286
[10141.440485] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10141.450506] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10141.457908] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10141.466870] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10141.475833] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10141.484767] Stack:
[10141.487974] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[10141.497272] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[10141.506555] ffff8800b2a37468 0000000020000000 010000000000000c 0000000000000000
[10141.515848] Call Trace:
[10141.519581] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[10141.528742] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10141.537899] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10141.546764] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10141.555433] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10141.562635] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10141.571982] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10141.578404] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10141.586684] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10141.593686] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10141.601953] Code: 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41 5c 41 5d 41 5e <41> 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0 0f 84 c2 fe ff ff 0f
[10169.196727] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10169.206353] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10169.338361] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10169.351646] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10169.367892] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10169.377273] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[10169.388944] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[10169.395892] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[10169.404871] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10169.413840] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[10169.422789] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[10169.431735] R13: 0000000000000286 R14: 0000000000000010 R15: ffffffffa121ba64
[10169.440685] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10169.450711] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10169.458116] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10169.467084] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10169.476051] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10169.484998] Stack:
[10169.488213] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[10169.497503] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[10169.506805] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[10169.516107] Call Trace:
[10169.519820] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[10169.528995] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10169.538154] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10169.547024] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10169.555700] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10169.562915] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10169.572281] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10169.578720] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10169.587025] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10169.594042] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10169.602318] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[10183.745240] INFO: rcu_sched self-detected stall on CPU { 24} (t=2760066 jiffies g=907112 c=907111 q=1553403)
[10183.757576] Task dump for CPU 24:
[10183.762313] OI_scrub R running task 0 5551 2 0x00000088
[10183.771234] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[10183.780564] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[10183.789870] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[10183.799149] Call Trace:
[10183.802831] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[10183.810815] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[10183.817776] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[10183.825317] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[10183.833021] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[10183.841290] [<ffffffff81099697>] update_process_times+0x47/0x80
[10183.848843] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[10183.856868] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[10183.864012] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[10183.871616] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[10183.878918] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[10183.885806] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[10183.893840] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[10183.901660] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[10183.909097] <EOI> [<ffffffffa1217d70>] ? osd_scrub_next+0x350/0x3c0 [osd_ldiskfs]
[10183.918432] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10183.927337] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[10183.936251] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10183.945152] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10183.953757] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10183.962159] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10183.969109] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10183.978206] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10183.984384] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10183.992401] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10183.999147] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10202.504635] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[10202.517650] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[10209.198127] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10209.207553] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10209.337628] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10209.350826] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10209.366902] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10209.376200] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[10209.388366] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[10209.395260] RAX: 0000000020000000 RBX: 0000000000000001 RCX: 0000000000000000
[10209.404200] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10209.413133] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10209.422063] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10209.430997] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[10209.439922] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10209.449927] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10209.457304] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10209.466250] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10209.475206] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10209.484146] Stack:
[10209.487363] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10209.496644] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10209.505931] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10209.515222] Call Trace:
[10209.518942] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10209.528106] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10209.536975] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10209.545639] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10209.552855] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10209.562223] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10209.568664] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10209.576950] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10209.583979] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10209.592271] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[10237.199102] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10237.208785] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10237.340666] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10237.353970] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10237.370258] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10237.379649] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[10237.391333] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[10237.398293] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[10237.407274] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10237.416256] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10237.425215] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10237.434169] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10237.443126] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10237.453155] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10237.460567] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10237.469532] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10237.478489] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10237.487434] Stack:
[10237.490651] ffff880703603d08 ffff880703603d57 ffff880703603df0 000000009f8f2001
[10237.499944] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[10237.509234] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[10237.518525] Call Trace:
[10237.522253] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10237.531424] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10237.540584] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10237.549459] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10237.558115] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10237.565318] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10237.574662] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10237.581086] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10237.589365] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10237.596378] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10237.604657] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[10265.200078] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10265.209776] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10265.341785] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10265.355076] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10265.371313] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10265.380696] RIP: 0010:[<ffffffffa121b9a2>] [<ffffffffa121b9a2>] osd_scrub_exec+0x12/0x5b0 [osd_ldiskfs]
[10265.392344] RSP: 0018:ffff880703603cf0 EFLAGS: 00000293
[10265.399282] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[10265.408258] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10265.417222] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10265.426171] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[10265.435120] R13: ffff880703603d08 R14: 0000000000000010 R15: ffffffffa121ba79
[10265.444067] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10265.454095] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10265.461501] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10265.470467] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10265.479429] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10265.488374] Stack:
[10265.491595] 0000000000000004 00000000ffffffff 0000000000000001 ffff880703603df0
[10265.500887] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[10265.510192] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[10265.519482] Call Trace:
[10265.523210] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10265.532378] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10265.541541] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10265.550415] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10265.559093] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10265.566308] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10265.575660] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10265.582093] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10265.590375] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10265.597389] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10265.605661] Code: 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <41> 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90
[10293.201053] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10293.210698] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10293.342808] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10293.356110] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10293.372398] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10293.381793] RIP: 0010:[<ffffffffa1217a29>] [<ffffffffa1217a29>] osd_scrub_next+0x9/0x3c0 [osd_ldiskfs]
[10293.393366] RSP: 0018:ffff880703603d08 EFLAGS: 00000287
[10293.400315] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[10293.409296] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10293.418258] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[10293.427229] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10293.436192] R13: ffff880703603d08 R14: ffff880703603c78 R15: 0000000000000246
[10293.445152] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10293.455196] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10293.462610] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10293.471583] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10293.480553] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10293.489502] Stack:
[10293.492725] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[10293.502029] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[10293.511333] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[10293.520633] Call Trace:
[10293.524368] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[10293.533544] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10293.542698] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10293.551557] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10293.560236] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10293.567441] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10293.576792] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10293.583226] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10293.591500] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10293.598514] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10293.606793] Code: 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 <41> 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb
[10316.493459] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[10316.508215] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[10321.202024] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10321.211646] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10321.343645] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10321.356930] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10321.373175] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10321.382558] RIP: 0010:[<ffffffffa1162b46>] [<ffffffffa1162b46>] ldiskfs_itable_unused_count+0x16/0x30 [ldiskfs]
[10321.395018] RSP: 0018:ffff880703603d08 EFLAGS: 00000246
[10321.401980] RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff88042bd78800
[10321.410973] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[10321.419957] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10321.428944] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10321.437936] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[10321.446927] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10321.456991] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10321.464427] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10321.473412] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10321.482389] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10321.491362] Stack:
[10321.494620] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[10321.503939] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[10321.513260] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[10321.522581] Call Trace:
[10321.526355] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[10321.535556] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10321.544725] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10321.553628] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10321.562332] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10321.569578] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10321.578955] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10321.585469] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10321.593789] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10321.600835] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10321.609143] Code: 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 <48> 83 39 3f 76 07 0f b7 46 32 c1 e0 10 09 d0 5d c3 66 0f 1f 84
[10349.202997] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10349.212708] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10349.344462] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10349.357748] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10349.373986] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10349.383366] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[10349.395016] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[10349.401962] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[10349.410935] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10349.419890] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10349.428839] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10349.437787] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10349.446739] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10349.456768] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10349.464175] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10349.473159] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10349.482122] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10349.491079] Stack:
[10349.494296] 0000000000000004 ffff880703603d57 ffff880703603df0 0000000000000000
[10349.503599] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[10349.512896] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff880419dc3a00
[10349.522200] Call Trace:
[10349.525921] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10349.535109] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10349.544281] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10349.553152] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10349.561838] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10349.569054] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10349.578416] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10349.584854] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10349.593155] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10349.600166] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10349.608443] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[10363.754505] INFO: rcu_sched self-detected stall on CPU { 24} (t=2940069 jiffies g=907112 c=907111 q=2019436)
[10363.766914] Task dump for CPU 24:
[10363.771645] OI_scrub R running task 0 5551 2 0x00000088
[10363.780554] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[10363.789878] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[10363.799189] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[10363.808479] Call Trace:
[10363.812161] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[10363.820135] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[10363.827101] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[10363.834630] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[10363.842328] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[10363.850582] [<ffffffff81099697>] update_process_times+0x47/0x80
[10363.858130] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[10363.866145] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[10363.873255] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[10363.880840] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[10363.888114] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[10363.894987] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[10363.903021] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[10363.910845] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[10363.918272] <EOI> [<ffffffffa121d1c3>] ? osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[10363.928104] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[10363.937207] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10363.946116] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10363.954738] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10363.963148] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10363.970098] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10363.979191] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10363.985362] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10363.993385] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10364.000141] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10389.204398] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10389.213823] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10389.344008] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10389.357209] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10389.373294] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10389.382606] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[10389.394168] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[10389.401034] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[10389.409943] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10389.418849] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10389.427752] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10389.436662] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10389.445565] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10389.455541] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10389.462893] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10389.471798] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10389.480710] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10389.489616] Stack:
[10389.492799] ffff880703603d08 ffff880703603d57 ffff880703603df0 000000009f8f2001
[10389.502049] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[10389.511320] 0000000000000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[10389.520587] Call Trace:
[10389.524291] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10389.533436] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10389.542575] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10389.551421] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10389.560079] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10389.567271] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10389.576596] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10389.582996] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10389.591258] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10389.598242] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10389.606490] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[10417.205373] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10417.215003] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10417.347255] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10417.360550] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10417.376843] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10417.386237] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[10417.397910] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[10417.404869] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[10417.413853] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10417.422831] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10417.431804] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10417.440761] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10417.449719] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10417.459754] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10417.467168] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10417.476146] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10417.485108] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10417.494063] Stack:
[10417.497276] ffff880703603d08 000000009f8f2001 ffff880703603df0 000000009f8f2001
[10417.506550] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[10417.515837] 0000000000000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[10417.525133] Call Trace:
[10417.528862] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10417.538032] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10417.547187] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10417.556051] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10417.564707] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10417.571918] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10417.581290] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10417.587722] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10417.596007] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10417.603018] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10417.611294] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[10445.206349] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10445.216014] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10445.348133] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10445.361426] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10445.377682] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10445.387069] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[10445.399318] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[10445.406263] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[10445.415246] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[10445.424217] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10445.433178] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10445.442133] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[10445.451087] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10445.461121] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10445.468532] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10445.477508] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10445.486467] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10445.495415] Stack:
[10445.498642] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10445.507945] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10445.517231] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10445.526515] Call Trace:
[10445.530244] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10445.539418] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10445.548299] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10445.556974] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10445.564208] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10445.573579] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10445.580018] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10445.588318] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10445.595339] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10445.603624] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[10473.207317] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10473.216972] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10473.349154] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10473.362468] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10473.378772] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10473.388185] RIP: 0010:[<ffffffffa121d1e9>] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10473.400446] RSP: 0018:ffff880703603d18 EFLAGS: 00000292
[10473.407417] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[10473.416401] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10473.425387] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10473.434355] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[10473.443312] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[10473.452270] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10473.462308] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10473.469720] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10473.478691] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10473.487667] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10473.496622] Stack:
[10473.499853] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10473.509154] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10473.518451] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10473.527749] Call Trace:
[10473.531479] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10473.540649] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10473.549533] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10473.558216] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10473.565434] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10473.574789] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10473.581223] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10473.589510] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10473.596526] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10473.604793] Code: 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 <85> c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 48 8b
[10501.208292] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10501.217946] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10501.350183] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10501.363509] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10501.379846] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10501.389266] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[10501.401519] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[10501.408487] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[10501.417480] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[10501.426466] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10501.435453] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[10501.444419] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[10501.453372] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10501.463412] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10501.470827] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10501.479812] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10501.488781] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10501.497730] Stack:
[10501.500953] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10501.510232] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10501.519537] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10501.528836] Call Trace:
[10501.532567] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10501.541743] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10501.550616] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10501.559308] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10501.566520] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10501.575875] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10501.582303] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10501.590584] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10501.597601] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10501.605879] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[10529.209275] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10529.218933] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10529.351295] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10529.364613] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10529.380931] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10529.390341] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[10529.402024] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[10529.408991] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[10529.417992] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10529.426979] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10529.435950] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10529.444909] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10529.453871] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10529.463905] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10529.471318] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10529.480292] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10529.489263] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10529.498232] Stack:
[10529.501476] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[10529.510775] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[10529.520068] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[10529.529373] Call Trace:
[10529.533130] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10529.542301] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10529.551473] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10529.560350] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10529.569031] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10529.576246] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10529.585622] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10529.592050] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10529.600349] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10529.607374] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10529.615657] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[10543.763775] INFO: rcu_sched self-detected stall on CPU { 24} (t=3120072 jiffies g=907112 c=907111 q=2916916)
[10543.776187] Task dump for CPU 24:
[10543.780929] OI_scrub R running task 0 5551 2 0x00000088
[10543.789862] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[10543.799254] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[10543.808560] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[10543.817835] Call Trace:
[10543.821516] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[10543.829490] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[10543.836455] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[10543.843986] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[10543.851675] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[10543.859926] [<ffffffff81099697>] update_process_times+0x47/0x80
[10543.867474] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[10543.875496] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[10543.882624] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[10543.890215] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[10543.897494] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[10543.904379] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[10543.912405] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[10543.920235] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[10543.927658] <EOI> [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10543.937285] [<ffffffffa121b9a4>] ? osd_scrub_exec+0x14/0x5b0 [osd_ldiskfs]
[10543.945803] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10543.954709] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10543.963621] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10543.972227] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10543.980641] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10543.987588] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10543.996678] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10544.002856] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10544.010873] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10544.017619] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10569.210661] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10569.220064] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10569.350004] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10569.363194] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10569.379239] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10569.388536] RIP: 0010:[<ffffffffa121b995>] [<ffffffffa121b995>] osd_scrub_exec+0x5/0x5b0 [osd_ldiskfs]
[10569.399998] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[10569.406866] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[10569.415762] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10569.424657] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10569.433552] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[10569.442440] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[10569.451331] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10569.461294] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10569.468638] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10569.477556] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10569.486461] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10569.495366] Stack:
[10569.498542] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[10569.507795] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[10569.517048] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[10569.526293] Call Trace:
[10569.529991] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10569.539325] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10569.548465] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10569.557314] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10569.565967] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10569.573148] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10569.582473] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10569.588880] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10569.597130] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10569.604115] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10569.612383] Code: 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 <55> 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83
[10597.211642] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10597.221250] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10597.353468] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10597.366768] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10597.383023] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10597.392411] RIP: 0010:[<ffffffffa121ba26>] [<ffffffffa121ba26>] osd_scrub_exec+0x96/0x5b0 [osd_ldiskfs]
[10597.404060] RSP: 0018:ffff880703603d10 EFLAGS: 00000292
[10597.411007] RAX: 0000000000000000 RBX: ffff880703603d57 RCX: 0000000000000000
[10597.419993] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10597.428968] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10597.437936] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10597.446892] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[10597.455850] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10597.465885] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10597.473301] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10597.482292] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10597.491259] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10597.500220] Stack:
[10597.503441] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[10597.512747] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[10597.522051] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[10597.531358] Call Trace:
[10597.535106] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10597.544472] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10597.553648] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10597.562522] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10597.571207] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10597.578426] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10597.587794] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10597.594242] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10597.602554] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10597.609575] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10597.617857] Code: 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f 5d <c3> 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d 88
[10625.212618] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10625.222284] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10625.354758] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10625.368065] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10625.384337] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10625.393743] RIP: 0010:[<ffffffffa121b9b3>] [<ffffffffa121b9b3>] osd_scrub_exec+0x23/0x5b0 [osd_ldiskfs]
[10625.405403] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[10625.412355] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[10625.421344] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10625.430321] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10625.439285] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10625.448237] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10625.457198] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10625.467238] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10625.474659] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10625.483641] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10625.492613] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10625.501568] Stack:
[10625.504780] ffff880703603d08 0000000000000001 ffffea00027e3c80 000000009f8f2001
[10625.514068] ffff880703603d57 ffff880703603d57 0000000000000000 0000000000000000
[10625.523373] ffff880703603d78 ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10
[10625.532666] Call Trace:
[10625.536400] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10625.545564] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10625.554722] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10625.563590] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10625.572258] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10625.579468] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10625.588830] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10625.595262] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10625.603547] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10625.610550] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10625.618827] Code: 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 <4c> 89 45 90 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8d
[10653.213594] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10653.223244] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10653.355075] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10653.368364] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10653.384630] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10653.394008] RIP: 0010:[<ffffffffa121d1e0>] [<ffffffffa121d1e0>] osd_inode_iteration+0x490/0xcc0 [osd_ldiskfs]
[10653.406239] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[10653.413185] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: 0000000000000000
[10653.422159] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10653.431131] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10653.440082] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10653.449028] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[10653.457968] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10653.467999] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10653.475410] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10653.484380] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10653.493338] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10653.502289] Stack:
[10653.505500] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10653.514795] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10653.524072] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10653.533369] Call Trace:
[10653.537080] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10653.546244] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10653.555114] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10653.563780] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10653.570986] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10653.580341] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10653.586778] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10653.595070] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10653.602088] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10653.610374] Code: 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff <48> 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff
[10681.214565] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10681.224245] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10681.356243] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10681.369552] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10681.385849] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10681.395254] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[10681.406922] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[10681.413882] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[10681.422882] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10681.431867] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10681.440835] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10681.449801] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10681.458756] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10681.468792] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10681.476203] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10681.485182] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10681.494150] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10681.503108] Stack:
[10681.506321] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[10681.515603] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[10681.524890] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[10681.534184] Call Trace:
[10681.537921] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10681.547087] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10681.556242] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10681.565115] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10681.573795] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10681.581003] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10681.590354] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10681.596783] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10681.605084] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10681.612096] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10681.620361] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[10709.215546] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10709.225199] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10709.357261] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10709.370544] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10709.386795] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10709.396167] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[10709.407812] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[10709.414747] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[10709.423727] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10709.432695] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10709.441654] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10709.450613] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10709.459577] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10709.469616] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10709.477030] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10709.486001] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10709.494953] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10709.503894] Stack:
[10709.507098] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[10709.516388] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[10709.525695] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[10709.534987] Call Trace:
[10709.538705] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10709.547878] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10709.557040] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10709.565911] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10709.574597] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10709.581815] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10709.591169] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10709.597601] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10709.605882] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10709.612893] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10709.621171] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[10723.773051] INFO: rcu_sched self-detected stall on CPU { 24} (t=3300075 jiffies g=907112 c=907111 q=3476409)
[10723.785411] Task dump for CPU 24:
[10723.790137] OI_scrub R running task 0 5551 2 0x00000088
[10723.799055] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[10723.808390] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[10723.817691] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[10723.826979] Call Trace:
[10723.830668] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[10723.838663] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[10723.845632] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[10723.853157] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[10723.860860] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[10723.869128] [<ffffffff81099697>] update_process_times+0x47/0x80
[10723.876685] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[10723.884705] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[10723.891831] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[10723.899422] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[10723.906695] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[10723.913575] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[10723.921600] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[10723.929411] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[10723.936848] <EOI> [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10723.946481] [<ffffffffa121b9a2>] ? osd_scrub_exec+0x12/0x5b0 [osd_ldiskfs]
[10723.954998] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10723.963914] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10723.972814] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10723.981423] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10723.989829] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10723.996776] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10724.005878] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10724.012054] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10724.020071] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10724.026822] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10749.216940] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10749.226394] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10749.356196] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10749.369377] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10749.385434] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10749.394712] RIP: 0010:[<ffffffffa121ba25>] [<ffffffffa121ba25>] osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[10749.406284] RSP: 0018:ffff880703603d08 EFLAGS: 00000292
[10749.413149] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[10749.422045] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10749.430956] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10749.439859] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10749.448748] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[10749.457639] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10749.467593] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10749.474927] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10749.483825] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10749.492727] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10749.501621] Stack:
[10749.504787] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[10749.514020] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[10749.523264] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[10749.532513] Call Trace:
[10749.536192] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10749.545322] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10749.554446] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10749.563282] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10749.571918] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10749.579086] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10749.588392] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10749.594795] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10749.603048] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10749.610027] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10749.618262] Code: 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d
[10777.217907] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10777.227572] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10777.359410] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10777.372704] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10777.388945] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10777.398329] RIP: 0010:[<ffffffffa1162b3d>] [<ffffffffa1162b3d>] ldiskfs_itable_unused_count+0xd/0x30 [ldiskfs]
[10777.410668] RSP: 0018:ffff880703603d08 EFLAGS: 00000286
[10777.417635] RAX: ffff8800b2a37468 RBX: ffff880703603d57 RCX: ffff88042bd78800
[10777.426607] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[10777.435574] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10777.444522] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10777.453471] R13: ffffea00027e3c80 R14: ffff880703603c70 R15: ffff880703603d08
[10777.462417] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10777.472428] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10777.479845] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10777.488811] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10777.497764] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10777.506711] Stack:
[10777.509920] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[10777.519205] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[10777.528504] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[10777.537804] Call Trace:
[10777.541540] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[10777.550906] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10777.560072] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10777.568935] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10777.577598] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10777.584811] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10777.594175] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10777.600614] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10777.608893] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10777.615903] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10777.624174] Code: 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 <31> c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76 07 0f b7 46 32 c1 e0
[10805.218891] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[10805.228550] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10805.360338] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10805.373668] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10805.389927] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10805.399323] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[10805.411565] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[10805.418512] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[10805.427501] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10805.436477] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10805.445438] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10805.454389] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[10805.463345] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10805.473379] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10805.480799] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10805.489771] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10805.498722] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10805.507669] Stack:
[10805.510881] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10805.520175] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10805.529483] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10805.538793] Call Trace:
[10805.542523] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10805.551691] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10805.560567] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10805.569240] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10805.576449] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10805.585795] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10805.592226] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10805.600512] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10805.607528] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10805.615802] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[10825.856229] perf: interrupt took too long (4030 > 4028), lowering kernel.perf_event_max_sample_rate to 49000
[10853.220561] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10853.230232] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10853.362727] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10853.376057] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10853.392348] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10853.401756] RIP: 0010:[<ffffffffa121ba5f>] [<ffffffffa121ba5f>] osd_scrub_exec+0xcf/0x5b0 [osd_ldiskfs]
[10853.413442] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[10853.420415] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[10853.429419] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10853.438410] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10853.447385] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10853.456352] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10853.465325] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10853.475369] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10853.482802] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10853.491800] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10853.500781] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10853.509741] Stack:
[10853.512982] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 ffff880703603d57
[10853.522307] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[10853.531631] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[10853.540920] Call Trace:
[10853.544665] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10853.553856] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10853.563041] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10853.571925] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10853.580631] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10853.587856] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10853.597234] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10853.603680] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10853.611992] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10853.619018] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10853.627313] Code: 10 00 00 44 89 4d 88 4c 89 ef e8 1d fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 <48> 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b
[10881.221544] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10881.231218] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10881.363455] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10881.376735] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10881.392986] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10881.402370] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[10881.414611] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[10881.421558] RAX: ffff8800b2a37468 RBX: ffff880703603d57 RCX: 0000000000000000
[10881.430540] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10881.439504] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10881.448455] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[10881.457403] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[10881.466353] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10881.476381] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10881.483785] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10881.492755] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10881.501710] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10881.510660] Stack:
[10881.513866] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10881.523160] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10881.532450] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10881.541731] Call Trace:
[10881.545460] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10881.554624] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10881.563492] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10881.572172] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10881.579383] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10881.588735] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10881.595164] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10881.603455] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10881.610472] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10881.618744] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[10903.782330] INFO: rcu_sched self-detected stall on CPU { 24} (t=3480078 jiffies g=907112 c=907111 q=4879055)
[10903.794692] Task dump for CPU 24:
[10903.799431] OI_scrub R running task 0 5551 2 0x00000088
[10903.808393] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[10903.817711] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[10903.827020] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[10903.836305] Call Trace:
[10903.840010] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[10903.848010] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[10903.854999] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[10903.862531] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[10903.870234] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[10903.878500] [<ffffffff81099697>] update_process_times+0x47/0x80
[10903.886064] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[10903.894086] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[10903.901218] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[10903.908818] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[10903.916107] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[10903.922986] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[10903.931021] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[10903.938847] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[10903.946269] <EOI> [<ffffffffa1217b6d>] ? osd_scrub_next+0x14d/0x3c0 [osd_ldiskfs]
[10903.955599] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[10903.964498] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10903.973395] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10903.981991] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10903.990398] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10903.997344] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10904.006425] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10904.012595] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10904.020605] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10904.027358] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10929.223216] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10929.232609] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10929.362863] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10929.376050] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10929.392125] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10929.401422] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[10929.413570] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[10929.420434] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[10929.429342] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10929.438237] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[10929.447136] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[10929.456040] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[10929.464941] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10929.474920] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10929.482269] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10929.491200] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10929.500105] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10929.509008] Stack:
[10929.512173] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[10929.521409] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[10929.530643] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[10929.539892] Call Trace:
[10929.543590] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10929.552731] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10929.561581] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10929.570226] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10929.577418] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10929.586740] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10929.593140] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10929.601391] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10929.608377] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10929.616632] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[10934.532800] Lustre: soaked-MDT0000: Client 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100) reconnecting
[10934.546060] Lustre: soaked-MDT0000: Connection restored to 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100)
[10957.224185] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10957.233862] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10957.365840] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10957.379151] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10957.395463] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10957.404874] RIP: 0010:[<ffffffffa1217a4b>] [<ffffffffa1217a4b>] osd_scrub_next+0x2b/0x3c0 [osd_ldiskfs]
[10957.416552] RSP: 0018:ffff880703603c70 EFLAGS: 00000282
[10957.423523] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: ffff880703603d98
[10957.432530] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10957.441524] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[10957.450512] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10957.459502] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10957.468494] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10957.478569] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10957.486013] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10957.495018] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10957.504009] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10957.512989] Stack:
[10957.516248] 000000009f8f2001 ffff880703603d08 000000009f8f2001 ffffea00027e3c80
[10957.525586] 000000009f8f2001 ffff880703603d57 0000000000000001 ffffffffa1217a20
[10957.534903] ffff880703603d98 ffff880703603d78 ffff8800b2a36000 ffff8803162f6000
[10957.544224] Call Trace:
[10957.547975] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10957.556881] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[10957.566071] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10957.575261] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10957.584149] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10957.592843] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10957.600079] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10957.609444] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10957.615903] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10957.624208] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10957.631249] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10957.639548] Code: 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87
[10985.225153] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[10985.234781] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[10985.366298] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[10985.379566] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[10985.395770] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[10985.405131] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[10985.416754] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[10985.423678] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[10985.432634] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[10985.441581] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[10985.450522] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[10985.459464] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[10985.468410] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[10985.478442] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[10985.485853] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[10985.494815] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[10985.503851] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[10985.512814] Stack:
[10985.516083] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[10985.525432] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[10985.534871] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff880419dc3a00
[10985.544158] Call Trace:
[10985.548004] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[10985.557178] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[10985.566344] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[10985.575206] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[10985.583990] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[10985.591337] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[10985.600697] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[10985.607437] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10985.615723] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[10985.622741] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[10985.631028] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[11013.226129] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11013.235811] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11013.367626] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11013.380910] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11013.397154] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11013.406543] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[11013.418203] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[11013.425139] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[11013.434104] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11013.443065] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[11013.452021] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11013.460968] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11013.469916] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11013.479941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11013.487342] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11013.496312] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11013.505264] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11013.514207] Stack:
[11013.517411] ffff880703603d08 000000009f8f2001 ffff880703603df0 000000009f8f2001
[11013.526688] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[11013.535988] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[11013.545266] Call Trace:
[11013.548974] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[11013.558133] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11013.567286] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11013.576139] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11013.584801] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11013.592008] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11013.601349] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11013.607771] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11013.616041] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11013.623052] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11013.631315] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[11041.227110] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11041.236799] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11041.368518] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11041.381807] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11041.398071] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11041.407788] RIP: 0010:[<ffffffffa121d1ee>] [<ffffffffa121d1ee>] osd_inode_iteration+0x49e/0xcc0 [osd_ldiskfs]
[11041.420373] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[11041.427512] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[11041.436592] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11041.445756] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11041.454914] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[11041.464162] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[11041.473191] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11041.483471] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11041.490902] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11041.499873] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11041.508967] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11041.518095] Stack:
[11041.521368] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11041.530688] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11041.539976] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11041.549268] Call Trace:
[11041.552999] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11041.562181] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11041.571057] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11041.579730] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11041.586933] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11041.596300] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11041.602744] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11041.611029] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11041.618047] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11041.626332] Code: 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 <75> 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 48 8b 85 50 ff ff ff
[11069.228080] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11069.237693] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11069.369611] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11069.382923] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11069.399242] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11069.408648] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[11069.420917] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[11069.427891] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[11069.436884] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11069.445867] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11069.454837] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11069.463792] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[11069.472743] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11069.482769] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11069.490187] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11069.499157] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11069.508128] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11069.517086] Stack:
[11069.520302] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11069.529600] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11069.538886] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11069.548188] Call Trace:
[11069.551908] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11069.561078] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11069.569957] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11069.578642] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11069.585854] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11069.595208] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11069.601638] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11069.609915] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11069.616926] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11069.625192] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[11083.791587] INFO: rcu_sched self-detected stall on CPU { 24} (t=3660081 jiffies g=907112 c=907111 q=5905472)
[11083.803940] Task dump for CPU 24:
[11083.808688] OI_scrub R running task 0 5551 2 0x00000088
[11083.817614] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[11083.826933] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[11083.836248] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[11083.845559] Call Trace:
[11083.849274] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[11083.857279] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[11083.864271] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[11083.871807] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[11083.879523] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[11083.887800] [<ffffffff81099697>] update_process_times+0x47/0x80
[11083.895370] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[11083.903400] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[11083.910541] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[11083.918142] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[11083.925425] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[11083.932322] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[11083.940362] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[11083.948205] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[11083.955629] <EOI> [<ffffffffa1217b89>] ? osd_scrub_next+0x169/0x3c0 [osd_ldiskfs]
[11083.964961] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11083.973862] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11083.982759] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11083.991362] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11083.999767] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11084.006714] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11084.015800] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11084.021960] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11084.029976] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11084.036734] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11109.229472] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11109.238868] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11109.368571] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11109.381759] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11109.397834] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11109.407133] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[11109.419303] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[11109.426166] RAX: ffff8800b2a37468 RBX: ffff880703603d98 RCX: 0000000000000000
[11109.435079] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11109.443987] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11109.452895] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[11109.461801] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[11109.470704] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11109.480670] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11109.488008] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11109.496935] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11109.505838] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11109.514725] Stack:
[11109.517886] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11109.527135] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11109.536396] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11109.545669] Call Trace:
[11109.549351] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11109.558476] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11109.567332] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11109.576096] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11109.583306] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11109.592618] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11109.599009] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11109.607248] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11109.614231] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11109.622468] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[11129.530033] Lustre: soaked-MDT0000: Connection restored to 7ea3a484-c027-1559-8163-dc85944c882c (at 192.168.1.143@o2ib100)
[11137.230449] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11137.240115] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11137.372046] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11137.385354] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11137.401670] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11137.411073] RIP: 0010:[<ffffffffa1217a4f>] [<ffffffffa1217a4f>] osd_scrub_next+0x2f/0x3c0 [osd_ldiskfs]
[11137.422755] RSP: 0018:ffff880703603c70 EFLAGS: 00000282
[11137.429712] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: ffff880703603d98
[11137.438708] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11137.447691] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11137.456658] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11137.465624] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11137.474591] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11137.484641] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11137.492075] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11137.501073] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11137.510061] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11137.519039] Stack:
[11137.522278] 000000009f8f2001 ffff880703603d08 ffff880703603d57 ffff880703603df0
[11137.531610] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[11137.540935] ffff880703603d57 0000000020000000 0000000000000000 ffff880703603d78
[11137.550268] Call Trace:
[11137.554016] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11137.563209] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11137.572399] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11137.581289] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11137.589989] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11137.597220] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11137.606604] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11137.613058] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11137.621370] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11137.628408] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11137.636704] Code: 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 <31> c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87 58 10 00 00
[11165.231430] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11165.241114] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11165.372816] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11165.386202] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11165.402458] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11165.411842] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[11165.423503] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[11165.430444] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[11165.439423] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11165.448387] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[11165.457336] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11165.466289] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11165.475241] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11165.485269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11165.492678] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11165.501641] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11165.510594] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11165.519535] Stack:
[11165.522751] ffffea00027e3c80 000000009f8f2001 0000000000000004 0000000000000001
[11165.532045] ffff880703603d57 ffff880703603d98 ffff880703603d78 ffff8800b2a36000
[11165.541378] ffff8803162f6000 ffffffffffffff10 ffffffffa1217a32 0000000000000010
[11165.550678] Call Trace:
[11165.554399] [<ffffffffa1217a32>] ? osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[11165.563182] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[11165.572349] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11165.581508] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11165.590369] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11165.599050] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11165.606259] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11165.615617] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11165.622049] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11165.630335] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11165.637346] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11165.645620] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[11193.232400] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11193.242056] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11193.373579] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11193.386865] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11193.403092] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11193.412466] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[11193.424098] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[11193.431040] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[11193.440009] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11193.448972] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11193.457916] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c70
[11193.466876] R13: ffff880703603d08 R14: 0000000000000010 R15: ffffffffa1217a4f
[11193.475840] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11193.485863] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11193.493274] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11193.502236] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11193.511190] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11193.520134] Stack:
[11193.523349] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[11193.532659] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[11193.541970] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[11193.551249] Call Trace:
[11193.554971] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11193.564144] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11193.573320] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11193.582190] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11193.590865] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11193.598076] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11193.607434] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11193.613874] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11193.622159] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11193.629172] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11193.637455] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[11221.233380] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11221.243073] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11221.375053] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11221.388344] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11221.404595] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11221.413983] RIP: 0010:[<ffffffffa121cfda>] [<ffffffffa121cfda>] osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[11221.426245] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[11221.433199] RAX: ffff88042bd78800 RBX: 0000000000000000 RCX: ffff88042bd78800
[11221.442182] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[11221.451161] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[11221.460132] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[11221.469101] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[11221.478061] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11221.488103] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11221.495515] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11221.504485] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11221.513446] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11221.522395] Stack:
[11221.525609] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11221.534909] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11221.544213] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11221.553503] Call Trace:
[11221.557230] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11221.566396] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11221.575267] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11221.583937] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11221.591145] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11221.600502] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11221.606934] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11221.615222] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11221.622237] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11221.630512] Code: a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff <48> 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[11249.234349] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11249.244012] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11249.375975] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11249.389300] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11249.405600] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11249.415012] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[11249.426698] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[11249.433659] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[11249.442648] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11249.451626] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11249.460592] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[11249.469545] R13: 0000000000000246 R14: 0000000000000010 R15: ffffffffa121ba12
[11249.478502] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11249.488534] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11249.495944] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11249.504904] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11249.513865] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11249.522818] Stack:
[11249.526027] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[11249.535324] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[11249.544624] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[11249.553934] Call Trace:
[11249.557660] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11249.566834] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11249.575995] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11249.584867] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11249.593538] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11249.600753] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11249.610106] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11249.616536] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11249.624820] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11249.631837] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11249.640104] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[11263.800871] INFO: rcu_sched self-detected stall on CPU { 24} (t=3840084 jiffies g=907112 c=907111 q=6877352)
[11263.813258] Task dump for CPU 24:
[11263.817996] OI_scrub R running task 0 5551 2 0x00000088
[11263.826928] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[11263.836243] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[11263.845556] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[11263.854834] Call Trace:
[11263.858516] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[11263.866478] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[11263.873446] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[11263.880972] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[11263.888671] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[11263.896921] [<ffffffff81099697>] update_process_times+0x47/0x80
[11263.904467] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[11263.912479] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[11263.919595] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[11263.927176] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[11263.934464] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[11263.941346] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[11263.949379] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[11263.957205] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[11263.964626] <EOI> [<ffffffffa1162b53>] ? ldiskfs_itable_unused_count+0x23/0x30 [ldiskfs]
[11263.974656] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[11263.983563] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11263.992465] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11264.001077] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11264.009484] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11264.016434] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11264.025520] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11264.031697] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11264.039726] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11264.046483] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11289.235743] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11289.245166] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11289.374875] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11289.388049] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11289.404115] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11289.413415] RIP: 0010:[<ffffffffa1217a29>] [<ffffffffa1217a29>] osd_scrub_next+0x9/0x3c0 [osd_ldiskfs]
[11289.424885] RSP: 0018:ffff880703603d08 EFLAGS: 00000287
[11289.431739] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[11289.440645] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11289.449537] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11289.458438] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11289.467339] R13: 0000000000000018 R14: ffff880703603c70 R15: 0000000000000282
[11289.476224] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11289.486179] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11289.493519] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11289.502418] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11289.511319] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11289.520205] Stack:
[11289.523361] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[11289.532571] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[11289.541814] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[11289.551072] Call Trace:
[11289.554757] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11289.563883] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11289.573005] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11289.581837] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11289.590472] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11289.597644] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11289.606960] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11289.613348] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11289.621586] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11289.628564] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11289.636796] Code: 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 <41> 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb
[11317.236725] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11317.246345] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11317.378088] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11317.391396] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11317.407642] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11317.417018] RIP: 0010:[<ffffffffa121d1c3>] [<ffffffffa121d1c3>] osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[11317.429271] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[11317.436206] RAX: 0000000000000004 RBX: ffff880703603d98 RCX: ffff880703603d98
[11317.445187] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[11317.454153] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[11317.463108] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[11317.472055] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[11317.480995] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11317.491014] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11317.498406] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11317.507368] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11317.516321] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11317.525259] Stack:
[11317.528464] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11317.537744] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11317.547032] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11317.556299] Call Trace:
[11317.560014] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11317.569176] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11317.578044] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11317.586712] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11317.593908] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11317.603253] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11317.609691] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11317.617973] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11317.624976] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11317.633240] Code: 00 00 48 c7 05 9b 86 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 <4c> 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff
[11332.823838] perf: interrupt took too long (5039 > 5037), lowering kernel.perf_event_max_sample_rate to 39000
[11357.238111] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11357.247813] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11357.379703] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11357.393006] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11357.409288] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11357.418687] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[11357.430343] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[11357.437288] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[11357.446279] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11357.455256] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[11357.464219] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11357.473186] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11357.482146] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11357.492188] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11357.499614] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11357.508605] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11357.517582] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11357.526554] Stack:
[11357.529785] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[11357.539108] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[11357.548426] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[11357.557744] Call Trace:
[11357.561479] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[11357.570670] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11357.579862] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11357.588743] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11357.597424] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11357.604655] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11357.614019] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11357.620465] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11357.628762] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11357.635792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11357.644092] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[11385.239093] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11385.248775] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11385.380726] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11385.394016] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11385.410258] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11385.419643] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[11385.431887] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[11385.438830] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[11385.447809] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[11385.456779] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11385.465737] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11385.474682] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[11385.483628] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11385.493658] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11385.501067] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11385.510034] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11385.518985] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11385.527923] Stack:
[11385.531137] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11385.540432] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11385.549724] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11385.559020] Call Trace:
[11385.562738] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11385.571898] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11385.580776] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11385.589443] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11385.596655] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11385.606008] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11385.612438] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11385.620730] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11385.627749] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11385.636027] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[11412.545528] Lustre: soaked-MDT0000: Client 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100) reconnecting
[11412.558814] Lustre: Skipped 1 previous similar message
[11412.565640] Lustre: soaked-MDT0000: Connection restored to 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100)
[11413.240063] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11413.249725] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11413.381777] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11413.395083] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11413.411375] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11413.420779] RIP: 0010:[<ffffffffa121d1d9>] [<ffffffffa121d1d9>] osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[11413.433038] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[11413.440021] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: 0000000000000000
[11413.449040] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11413.458052] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11413.467053] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11413.476056] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[11413.485061] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11413.495150] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11413.502607] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11413.511623] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11413.520626] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11413.529628] Stack:
[11413.532900] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11413.542247] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11413.551589] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11413.560931] Call Trace:
[11413.564699] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11413.573912] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11413.582835] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11413.591556] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11413.598818] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11413.608224] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11413.614712] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11413.623037] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11413.630102] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11413.638444] Code: 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5
[11441.241045] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11441.250738] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11441.382767] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11441.396077] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11441.412387] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11441.421789] RIP: 0010:[<ffffffffa1217a4b>] [<ffffffffa1217a4b>] osd_scrub_next+0x2b/0x3c0 [osd_ldiskfs]
[11441.433457] RSP: 0018:ffff880703603c70 EFLAGS: 00000282
[11441.440425] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: ffff880703603d98
[11441.449417] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11441.458394] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11441.467362] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11441.476323] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11441.485278] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11441.495294] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11441.502700] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11441.511666] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11441.520631] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11441.529586] Stack:
[11441.532791] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[11441.542082] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[11441.551374] ffff880703603d57 ffff8800b2a37468 0000000000000000 ffff880703603d78
[11441.560663] Call Trace:
[11441.564387] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11441.573556] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11441.582716] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11441.591583] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11441.600247] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11441.607458] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11441.616799] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11441.623232] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11441.631503] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11441.638507] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11441.646780] Code: 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87
[11443.810127] INFO: rcu_sched self-detected stall on CPU { 24} (t=4020087 jiffies g=907112 c=907111 q=7512209)
[11443.822505] Task dump for CPU 24:
[11443.827223] OI_scrub R running task 0 5551 2 0x00000088
[11443.836145] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[11443.845475] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[11443.854763] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[11443.864038] Call Trace:
[11443.867713] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[11443.875685] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[11443.882643] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[11443.890157] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[11443.897838] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[11443.906084] [<ffffffff81099697>] update_process_times+0x47/0x80
[11443.913626] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[11443.921641] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[11443.928766] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[11443.936345] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[11443.943613] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[11443.950482] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[11443.958503] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[11443.966325] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[11443.973744] <EOI> [<ffffffffa121cfc1>] ? osd_inode_iteration+0x271/0xcc0 [osd_ldiskfs]
[11443.983561] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[11443.992658] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11444.001558] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11444.010166] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11444.018568] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11444.025510] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11444.034601] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11444.040765] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11444.048797] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11444.055544] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11469.242020] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11469.251457] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11469.381350] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11469.394537] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11469.410632] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11469.419936] RIP: 0010:[<ffffffffa1217a2b>] [<ffffffffa1217a2b>] osd_scrub_next+0xb/0x3c0 [osd_ldiskfs]
[11469.431398] RSP: 0018:ffff880703603d00 EFLAGS: 00000287
[11469.438257] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[11469.447162] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11469.456076] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11469.464972] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11469.473872] R13: ffff880703603d08 R14: ffff880703603c70 R15: 0000000000000282
[11469.482761] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11469.492727] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11469.500074] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11469.508985] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11469.517879] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11469.526781] Stack:
[11469.529960] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[11469.539228] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[11469.548478] ffff8800b2a37468 0000000020000000 010000000000000c 0000000000000000
[11469.557742] Call Trace:
[11469.561434] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11469.570571] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11469.579705] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11469.588545] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11469.597198] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11469.604385] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11469.613705] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11469.620099] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11469.628350] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11469.635325] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11469.643563] Code: 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 <49> 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83
[11497.242988] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11497.252635] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11497.384482] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11497.397767] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11497.414014] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11497.423395] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[11497.435049] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[11497.441984] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[11497.450958] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11497.459919] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[11497.468867] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11497.477814] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11497.486762] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11497.496788] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11497.504198] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11497.513164] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11497.522119] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11497.531057] Stack:
[11497.534266] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[11497.543569] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[11497.552859] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[11497.562149] Call Trace:
[11497.565869] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[11497.575043] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11497.584217] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11497.593069] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11497.601746] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11497.608962] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11497.618314] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11497.624757] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11497.633029] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11497.640045] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11497.648334] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[11525.243964] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11525.253644] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11525.385519] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11525.398806] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11525.415079] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11525.424468] RIP: 0010:[<ffffffffa121cfbd>] [<ffffffffa121cfbd>] osd_inode_iteration+0x26d/0xcc0 [osd_ldiskfs]
[11525.436708] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[11525.443654] RAX: 00000000000072ce RBX: 000000009f8f2001 RCX: ffff88042bd78800
[11525.452633] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[11525.461601] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11525.470561] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[11525.479509] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[11525.488455] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11525.498485] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11525.505891] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11525.514865] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11525.523814] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11525.532750] Stack:
[11525.535957] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11525.545248] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11525.554543] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11525.563842] Call Trace:
[11525.567565] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11525.576721] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11525.585593] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11525.594255] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11525.601463] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11525.610811] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11525.617246] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11525.625516] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11525.632522] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11525.640795] Code: f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 <48> 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00
[11553.244946] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11553.254612] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11553.386621] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11553.399926] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11553.416220] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11553.425624] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[11553.437863] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[11553.444826] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[11553.453811] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[11553.462788] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11553.471747] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[11553.480704] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[11553.489659] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11553.499693] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11553.507106] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11553.516075] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11553.525051] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11553.534010] Stack:
[11553.537229] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11553.546523] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11553.555813] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11553.565116] Call Trace:
[11553.568860] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11553.578038] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11553.586914] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11553.595608] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11553.602834] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11553.612185] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11553.618626] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11553.626917] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11553.633941] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11553.642216] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[11581.245926] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11581.255597] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11581.388183] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11581.401508] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11581.417832] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11581.427254] RIP: 0010:[<ffffffffa121d00e>] [<ffffffffa121d00e>] osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[11581.439521] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[11581.446496] RAX: 0000000000000004 RBX: ffff880703603d98 RCX: ffff880703603d98
[11581.455496] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[11581.464495] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[11581.473469] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[11581.482432] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[11581.491388] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11581.501411] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11581.508822] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11581.517798] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11581.526762] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11581.535719] Stack:
[11581.538934] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11581.548237] ffff8803162f6000 ffff8800b2a37468[11581.551605] Lustre: soaked-MDT0000: Connection restored to 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100)
0000000020000000 010000000000000c
[11581.571917] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11581.581232] Call Trace:
[11581.584988] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11581.594185] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11581.603084] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11581.611784] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11581.619025] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11581.628403] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11581.634866] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11581.643171] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11581.650210] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11581.658511] Code: 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 <66> 90 0f 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90
[11609.246890] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11609.256565] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11609.388541] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11609.401846] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11609.418139] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11609.427539] RIP: 0010:[<ffffffffa1217b6d>] [<ffffffffa1217b6d>] osd_scrub_next+0x14d/0x3c0 [osd_ldiskfs]
[11609.439303] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[11609.446267] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[11609.455263] RDX: ffff880703603d78 RSI: 00000000fb6db880 RDI: ffff8803162f6000
[11609.464250] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11609.473243] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11609.482223] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11609.491212] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11609.501275] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11609.508717] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11609.517729] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11609.526721] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11609.535699] Stack:
[11609.538949] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[11609.548297] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[11609.557629] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[11609.566939] Call Trace:
[11609.570685] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11609.579853] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11609.589016] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11609.597879] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11609.606544] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11609.613745] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11609.623086] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11609.629515] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11609.637791] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11609.644802] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11609.653063] Code: 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 <65> 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41
[11623.819412] INFO: rcu_sched self-detected stall on CPU { 24} (t=4200090 jiffies g=907112 c=907111 q=8013065)
[11623.831774] Task dump for CPU 24:
[11623.836517] OI_scrub R running task 0 5551 2 0x00000088
[11623.845430] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[11623.854745] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[11623.864041] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[11623.873308] Call Trace:
[11623.876982] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[11623.884951] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[11623.891914] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[11623.899437] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[11623.907131] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[11623.915390] [<ffffffff81099697>] update_process_times+0x47/0x80
[11623.922940] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[11623.930953] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[11623.938091] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[11623.945675] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[11623.952957] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[11623.959832] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[11623.967857] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[11623.975670] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[11623.983087] <EOI> [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11623.992705] [<ffffffffa121b9a4>] ? osd_scrub_exec+0x14/0x5b0 [osd_ldiskfs]
[11624.001223] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[11624.010119] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11624.019023] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11624.027627] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11624.036028] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11624.042968] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11624.052074] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11624.058252] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11624.066278] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11624.073032] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11649.248288] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11649.257684] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11649.387709] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11649.400884] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11649.416927] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11649.426214] RIP: 0010:[<ffffffffa1217a3b>] [<ffffffffa1217a3b>] osd_scrub_next+0x1b/0x3c0 [osd_ldiskfs]
[11649.437776] RSP: 0018:ffff880703603ce0 EFLAGS: 00000287
[11649.444639] RAX: ffffffffa1217a20 RBX: ffff880703603c70 RCX: ffff880703603d98
[11649.453542] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11649.462445] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11649.471338] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[11649.480237] R13: ffffffffa1217b6d R14: ffffffffffffff10 R15: ffff8803162f6000
[11649.489129] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11649.499096] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11649.506445] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11649.515371] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11649.524283] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11649.533175] Stack:
[11649.536354] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[11649.545619] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[11649.554883] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[11649.564119] Call Trace:
[11649.567819] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11649.576965] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11649.586100] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11649.594948] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11649.603594] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11649.610769] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11649.620093] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11649.626498] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11649.634757] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11649.641744] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11649.649994] Code: 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 <48> 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31
[11677.249258] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11677.258874] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11677.390612] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11677.403910] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11677.420178] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11677.429562] RIP: 0010:[<ffffffffa121d1d9>] [<ffffffffa121d1d9>] osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[11677.441813] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[11677.448762] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: 0000000000000000
[11677.457747] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11677.466722] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11677.475685] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11677.484643] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[11677.493603] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11677.503647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11677.511063] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11677.520037] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11677.529004] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11677.537957] Stack:
[11677.541167] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11677.550460] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11677.559762] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11677.569039] Call Trace:
[11677.572761] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11677.581931] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11677.590807] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11677.599483] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11677.606694] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11677.616060] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11677.622497] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11677.630794] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11677.637812] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11677.646095] Code: 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5
[11705.250235] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11705.259925] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11705.391818] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11705.405125] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11705.421421] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11705.430816] RIP: 0010:[<ffffffffa121cffc>] [<ffffffffa121cffc>] osd_inode_iteration+0x2ac/0xcc0 [osd_ldiskfs]
[11705.443069] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[11705.450031] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[11705.459011] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[11705.467991] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[11705.476960] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[11705.485914] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[11705.494857] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11705.504876] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11705.512279] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11705.521237] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11705.530194] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11705.539143] Stack:
[11705.542351] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11705.551644] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11705.560929] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11705.570232] Call Trace:
[11705.573949] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11705.583110] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11705.591976] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11705.600647] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11705.607855] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11705.617221] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11705.623644] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11705.631923] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11705.638938] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11705.647212] Code: 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 <41> 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 0f
[11733.251217] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11733.260874] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11733.392776] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11733.406078] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11733.422367] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11733.431773] RIP: 0010:[<ffffffffa121cfda>] [<ffffffffa121cfda>] osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[11733.444028] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[11733.450986] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[11733.459970] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[11733.468946] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[11733.477916] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11733.486868] R13: ffff880703603df0 R14: 000000009f8f2001 R15: 0000000000000018
[11733.495827] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11733.505855] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11733.513283] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11733.522257] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11733.531230] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11733.540186] Stack:
[11733.543407] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11733.552703] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11733.562008] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11733.571322] Call Trace:
[11733.575061] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11733.584235] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11733.593116] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11733.601803] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11733.609030] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11733.618393] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11733.624830] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11733.633130] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11733.640140] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11733.648417] Code: a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff <48> 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[11761.252185] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11761.261851] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11761.393804] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11761.407118] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11761.423439] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11761.432847] RIP: 0010:[<ffffffffa121ba64>] [<ffffffffa121ba64>] osd_scrub_exec+0xd4/0x5b0 [osd_ldiskfs]
[11761.444517] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[11761.451489] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[11761.460481] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11761.469464] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[11761.478430] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11761.487386] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11761.496337] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11761.506374] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11761.513779] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11761.522754] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11761.531717] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11761.540663] Stack:
[11761.543883] 0000000000000004 0000000000000001 0000000000000004 ffff880703603d98
[11761.553170] ffff880703603d57 0000000000000000 ffff8803162f6000 ffffffffffffff10
[11761.562455] ffffffffa1217b80 0000000000000010 0000000000000282 ffff880703603ce0
[11761.571750] Call Trace:
[11761.575477] [<ffffffffa1217b80>] ? osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[11761.584344] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[11761.593508] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11761.602667] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11761.611529] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11761.620192] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11761.627395] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11761.636738] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11761.643171] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11761.651450] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11761.658456] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11761.666721] Code: 4d 88 4c 89 ef e8 1d fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 <f6> 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00
[11789.253161] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11789.262830] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11789.394327] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11789.407600] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11789.423807] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11789.433180] RIP: 0010:[<ffffffffa1162b53>] [<ffffffffa1162b53>] ldiskfs_itable_unused_count+0x23/0x30 [ldiskfs]
[11789.445602] RSP: 0018:ffff880703603d08 EFLAGS: 00000297
[11789.452535] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[11789.461501] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[11789.470453] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[11789.479403] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11789.488352] R13: ffff880703603d08 R14: ffff880703603c70 R15: 0000000000000282
[11789.497304] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11789.507333] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11789.514745] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11789.523700] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11789.532649] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11789.541586] Stack:
[11789.544788] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[11789.554083] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[11789.563376] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[11789.572667] Call Trace:
[11789.576394] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[11789.585550] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11789.594704] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11789.603562] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11789.612221] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11789.619429] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11789.628775] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11789.635219] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11789.643500] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11789.650509] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11789.658783] Code: 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76 07 0f b7 46 32 c1 e0 10 <09> d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 89 16
[11803.828669] INFO: rcu_sched self-detected stall on CPU { 24} (t=4380093 jiffies g=907112 c=907111 q=8604937)
[11803.841047] Task dump for CPU 24:
[11803.845787] OI_scrub R running task 0 5551 2 0x00000088
[11803.854725] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[11803.864043] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[11803.873358] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[11803.882648] Call Trace:
[11803.886331] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[11803.894318] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[11803.901280] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[11803.908793] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[11803.916479] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[11803.924731] [<ffffffff81099697>] update_process_times+0x47/0x80
[11803.932270] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[11803.940290] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[11803.947407] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[11803.954993] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[11803.962266] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[11803.969139] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[11803.977162] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[11803.984992] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[11803.992415] <EOI> [<ffffffffa121d00e>] ? osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[11804.002231] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11804.011338] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11804.020243] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11804.028857] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11804.037264] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11804.044214] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11804.053300] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11804.059469] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11804.067490] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11804.074244] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11829.254554] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11829.263999] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11829.393719] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11829.406897] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11829.422975] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11829.432271] RIP: 0010:[<ffffffffa1217b6d>] [<ffffffffa1217b6d>] osd_scrub_next+0x14d/0x3c0 [osd_ldiskfs]
[11829.443935] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[11829.450794] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[11829.459690] RDX: ffff880703603d78 RSI: 00000000fb6db880 RDI: ffff8803162f6000
[11829.468590] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11829.477487] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11829.486386] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11829.495281] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11829.505240] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11829.512586] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11829.521493] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11829.530402] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11829.539301] Stack:
[11829.542472] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[11829.551729] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[11829.560971] ffff880703603d57 0000000000000000 0000000000000000 ffff880703603d78
[11829.570219] Call Trace:
[11829.573910] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11829.583051] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11829.592188] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11829.601026] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11829.609671] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11829.616866] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11829.626179] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11829.632575] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11829.640818] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11829.647794] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11829.656032] Code: 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 <65> 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41
[11857.255529] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[11857.265210] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11857.396889] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11857.410172] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11857.426417] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11857.435796] RIP: 0010:[<ffffffffa121cfda>] [<ffffffffa121cfda>] osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[11857.448035] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[11857.454984] RAX: ffff88042bd78800 RBX: 0000000000000001 RCX: ffff88042bd78800
[11857.463968] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[11857.472931] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[11857.481886] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11857.490837] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[11857.499795] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11857.509827] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11857.517238] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11857.526207] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11857.535166] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11857.544112] Stack:
[11857.547315] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11857.556612] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11857.565914] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11857.575203] Call Trace:
[11857.578923] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11857.588103] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11857.596985] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11857.605668] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11857.612883] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11857.622240] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11857.628678] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11857.636970] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11857.643990] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11857.652268] Code: a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff <48> 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[11885.256505] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11885.266132] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11885.398122] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11885.411444] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11885.427748] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11885.437159] RIP: 0010:[<ffffffffa1217b69>] [<ffffffffa1217b69>] osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[11885.448931] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[11885.455898] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[11885.464897] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[11885.473887] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[11885.482855] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[11885.491824] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[11885.500780] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11885.510815] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11885.518227] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11885.527196] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11885.536161] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11885.545110] Stack:
[11885.548325] 000000009f8f2001 0000000000000004 0000000000000001 0000000000000004
[11885.557607] ffff880703603d98 ffff880703603d57 0000000000000000 ffff8803162f6000
[11885.566902] ffffffffffffff10 ffffffffa1217b80 0000000000000010 0000000000000282
[11885.576198] Call Trace:
[11885.579930] [<ffffffffa1217b80>] ? osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[11885.588811] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[11885.597978] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11885.607142] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11885.616016] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11885.624681] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11885.631889] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11885.641245] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11885.647674] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11885.655956] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11885.662963] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11885.671223] Code: 45 90 74 49 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 <48> 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83
[11911.485625] Lustre: soaked-MDT0000: Connection restored to 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100)
[11913.257481] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11913.267187] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11913.398846] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11913.412131] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11913.428365] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11913.437745] RIP: 0010:[<ffffffffa1162b36>] [<ffffffffa1162b36>] ldiskfs_itable_unused_count+0x6/0x30 [ldiskfs]
[11913.450082] RSP: 0018:ffff880703603d08 EFLAGS: 00000286
[11913.457032] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[11913.466018] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[11913.475000] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[11913.483970] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[11913.492951] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[11913.501931] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11913.511971] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11913.519381] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11913.528353] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11913.537321] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11913.546285] Stack:
[11913.549537] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[11913.558831] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[11913.568130] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[11913.577427] Call Trace:
[11913.581179] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[11913.590554] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11913.599731] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11913.608621] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11913.617315] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11913.624559] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11913.633932] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11913.640386] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11913.648696] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11913.655727] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11913.664022] Code: 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 <48> 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76
[11941.258456] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11941.268116] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11941.399876] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11941.413170] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11941.429433] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11941.438816] RIP: 0010:[<ffffffffa121d1c3>] [<ffffffffa121d1c3>] osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[11941.451057] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[11941.458005] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[11941.466986] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[11941.475948] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[11941.484895] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[11941.493843] R13: ffff880703603df0 R14: 000000009f8f2001 R15: ffffea00027e3c80
[11941.502793] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11941.512824] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11941.520219] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11941.529192] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11941.538151] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11941.547114] Stack:
[11941.550317] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11941.559612] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11941.568917] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11941.578217] Call Trace:
[11941.581928] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11941.591095] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11941.599968] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11941.608646] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11941.615860] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11941.625212] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11941.631642] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11941.639920] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11941.646931] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11941.655208] Code: 00 00 48 c7 05 9b 86 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 <4c> 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff
[11969.259438] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[11969.269078] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[11969.401075] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[11969.414380] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[11969.430682] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[11969.440091] RIP: 0010:[<ffffffffa121d1c3>] [<ffffffffa121d1c3>] osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[11969.452346] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[11969.459314] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[11969.468307] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[11969.477291] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[11969.486249] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[11969.495200] R13: ffff880703603df0 R14: 000000009f8f2001 R15: 0000000000000018
[11969.504149] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[11969.514174] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[11969.521590] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[11969.530567] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[11969.539534] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[11969.548483] Stack:
[11969.551701] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[11969.560994] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[11969.570278] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[11969.579595] Call Trace:
[11969.583326] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11969.592500] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11969.601373] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11969.610054] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11969.617281] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11969.626639] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11969.633070] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11969.641359] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11969.648379] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11969.656654] Code: 00 00 48 c7 05 9b 86 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 <4c> 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff
[11983.837939] INFO: rcu_sched self-detected stall on CPU { 24} (t=4560096 jiffies g=907112 c=907111 q=9683906)
[11983.850317] Task dump for CPU 24:
[11983.855078] OI_scrub R running task 0 5551 2 0x00000088
[11983.864020] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[11983.873340] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[11983.882678] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[11983.891984] Call Trace:
[11983.895688] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[11983.903681] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[11983.910657] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[11983.918202] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[11983.925908] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[11983.934185] [<ffffffff81099697>] update_process_times+0x47/0x80
[11983.941761] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[11983.949792] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[11983.956927] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[11983.964536] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[11983.971830] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[11983.978726] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[11983.986780] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[11983.994629] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[11984.002065] <EOI> [<ffffffffa121cfbd>] ? osd_inode_iteration+0x26d/0xcc0 [osd_ldiskfs]
[11984.011886] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[11984.020984] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[11984.029893] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[11984.038509] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[11984.046923] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[11984.053883] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[11984.062973] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[11984.069149] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[11984.077176] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[11984.083943] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12009.260833] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12009.270264] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12009.400034] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12009.413214] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12009.429289] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12009.438585] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[12009.450733] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[12009.457590] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[12009.466498] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12009.475408] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12009.484306] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[12009.493206] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12009.502103] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12009.512067] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12009.519409] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12009.528311] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12009.537215] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12009.546110] Stack:
[12009.549274] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12009.558498] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12009.567749] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12009.577023] Call Trace:
[12009.580718] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12009.589863] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12009.598709] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12009.607365] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12009.614557] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12009.623894] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12009.630290] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12009.638534] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12009.645517] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12009.653764] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[12037.261800] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12037.271477] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12037.403502] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12037.416808] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12037.433108] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12037.442509] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[12037.454189] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[12037.461141] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[12037.470132] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12037.479113] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[12037.488064] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12037.497015] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12037.505961] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12037.515985] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12037.523390] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12037.532356] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12037.541321] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12037.550271] Stack:
[12037.553486] 000000009f8f2001 ffff880703603d08 ffff880703603d57 ffff880703603df0
[12037.562769] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[12037.572069] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[12037.581379] Call Trace:
[12037.585103] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[12037.594271] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12037.603437] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12037.612306] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12037.620970] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12037.628174] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12037.637518] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12037.643946] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12037.652227] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12037.659232] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12037.667496] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[12065.262776] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12065.272470] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12065.404143] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12065.417427] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12065.433670] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12065.443051] RIP: 0010:[<ffffffffa121d1fc>] [<ffffffffa121d1fc>] osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[12065.455282] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[12065.462227] RAX: 0000000020000000 RBX: 0000000000000001 RCX: 0000000000000000
[12065.471201] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12065.480165] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12065.489122] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12065.498071] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[12065.507020] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12065.517046] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12065.524451] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12065.533414] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12065.542358] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12065.551296] Stack:
[12065.554501] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12065.563784] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12065.573083] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12065.582355] Call Trace:
[12065.586069] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12065.595233] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12065.604104] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12065.612774] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12065.619981] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12065.629346] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12065.635776] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12065.644055] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12065.651070] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12065.659351] Code: 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 <48> 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2
[12078.568324] Lustre: soaked-MDT0000: Client af5c0dbc-bc84-dcde-3672-645b97702b86 (at 192.168.1.122@o2ib100) reconnecting
[12078.583209] Lustre: Skipped 2 previous similar messages
[12093.263752] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12093.273463] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12093.405225] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12093.418520] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12093.434800] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12093.444195] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[12093.456444] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[12093.463407] RAX: ffff8800b2a37468 RBX: ffff880703603d98 RCX: 0000000000000000
[12093.472402] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12093.481391] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12093.490374] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[12093.499355] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[12093.508335] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12093.518379] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12093.525807] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12093.534783] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12093.543764] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12093.552727] Stack:
[12093.555960] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12093.565270] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12093.574579] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12093.583887] Call Trace:
[12093.587626] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12093.596822] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12093.605706] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12093.614387] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12093.621614] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12093.630969] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12093.637420] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12093.645713] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12093.652738] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12093.661019] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[12121.264726] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12121.274436] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12121.406348] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12121.419646] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12121.435923] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12121.445317] RIP: 0010:[<ffffffffa121d1c3>] [<ffffffffa121d1c3>] osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[12121.457556] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[12121.464516] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[12121.473496] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[12121.482465] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[12121.491416] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[12121.500355] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12121.509294] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12121.519313] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12121.526714] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12121.535673] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12121.544634] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12121.553583] Stack:
[12121.556798] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12121.566094] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12121.575384] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12121.584697] Call Trace:
[12121.588422] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12121.597590] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12121.606470] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12121.615159] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12121.622374] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12121.631722] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12121.638156] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12121.646434] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12121.653446] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12121.661726] Code: 00 00 48 c7 05 9b 86 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 <4c> 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff
[12149.265708] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12149.275384] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12149.407428] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12149.420739] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12149.437033] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12149.446437] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[12149.458695] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[12149.465666] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[12149.474656] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[12149.483632] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12149.492605] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12149.501569] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12149.510513] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12149.520544] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12149.527948] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12149.536910] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12149.545875] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12149.554830] Stack:
[12149.558051] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12149.567344] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12149.576644] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12149.585973] Call Trace:
[12149.589701] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12149.598872] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12149.607747] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12149.616427] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12149.623643] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12149.632992] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12149.639431] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12149.647709] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12149.654730] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12149.663006] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[12163.847209] INFO: rcu_sched self-detected stall on CPU { 24} (t=4740099 jiffies g=907112 c=907111 q=10773653)
[12163.859658] Task dump for CPU 24:
[12163.864408] OI_scrub R running task 0 5551 2 0x00000088
[12163.873335] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[12163.882668] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[12163.891978] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[12163.901285] Call Trace:
[12163.904985] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[12163.912979] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[12163.919954] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[12163.927496] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[12163.935200] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[12163.943468] [<ffffffff81099697>] update_process_times+0x47/0x80
[12163.951037] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[12163.959069] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[12163.966195] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[12163.973801] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[12163.981098] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[12163.987982] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[12163.996024] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[12164.003852] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[12164.011284] <EOI> [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12164.020901] [<ffffffffa121b999>] ? osd_scrub_exec+0x9/0x5b0 [osd_ldiskfs]
[12164.029318] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12164.038222] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12164.047123] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12164.055742] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12164.064144] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12164.071087] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12164.080167] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12164.086334] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12164.094355] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12164.101106] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12189.267096] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12189.276529] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12189.406129] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12189.419305] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12189.435345] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12189.444628] RIP: 0010:[<ffffffffa1162b46>] [<ffffffffa1162b46>] ldiskfs_itable_unused_count+0x16/0x30 [ldiskfs]
[12189.456954] RSP: 0018:ffff880703603d08 EFLAGS: 00000246
[12189.463814] RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff88042bd78800
[12189.472711] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[12189.481683] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12189.490582] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12189.499473] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[12189.508358] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12189.518305] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12189.525643] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12189.534552] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12189.543449] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12189.552337] Stack:
[12189.555497] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[12189.564733] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[12189.573981] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[12189.583238] Call Trace:
[12189.586933] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[12189.596069] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12189.605196] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12189.614033] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12189.622665] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12189.629831] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12189.639135] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12189.645525] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12189.653757] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12189.660731] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12189.668965] Code: 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 <48> 83 39 3f 76 07 0f b7 46 32 c1 e0 10 09 d0 5d c3 66 0f 1f 84
[12217.268072] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12217.277695] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12217.409406] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12217.422696] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12217.438951] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12217.448329] RIP: 0010:[<ffffffffa121d00e>] [<ffffffffa121d00e>] osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[12217.460563] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[12217.467500] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[12217.476481] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[12217.485454] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[12217.494409] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[12217.503341] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12217.512287] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12217.522307] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12217.529715] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12217.538676] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12217.547633] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12217.556575] Stack:
[12217.559783] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12217.569064] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12217.578378] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12217.587670] Call Trace:
[12217.591393] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12217.600555] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12217.609427] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12217.618107] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12217.625319] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12217.634667] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12217.641098] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12217.649382] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12217.656385] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12217.664657] Code: 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 <66> 90 0f 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90
[12245.269046] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12245.278709] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12245.410609] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12245.423919] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12245.440210] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12245.449618] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[12245.461873] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[12245.468836] RAX: ffff88042bd78800 RBX: ffff880703603d98 RCX: ffff88042bd78800
[12245.477826] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[12245.486806] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12245.495788] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[12245.504740] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[12245.513687] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12245.523707] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12245.531110] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12245.540069] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12245.549024] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12245.557964] Stack:
[12245.561168] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12245.570441] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12245.579727] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12245.589019] Call Trace:
[12245.592742] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12245.601901] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12245.610767] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12245.619436] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12245.626643] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12245.635982] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12245.642404] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12245.650675] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12245.657680] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12245.665945] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[12273.270021] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12273.279687] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12273.411523] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12273.424827] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12273.441103] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12273.450509] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[12273.462751] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[12273.469708] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[12273.478690] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[12273.487671] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12273.496631] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12273.505579] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12273.514526] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12273.524540] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12273.531943] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12273.540898] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12273.549861] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12273.558813] Stack:
[12273.562035] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12273.571327] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12273.580616] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12273.589918] Call Trace:
[12273.593645] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12273.602813] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12273.611690] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12273.620378] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12273.627596] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12273.636938] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12273.643374] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12273.651657] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12273.658672] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12273.666947] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[12301.270998] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12301.280618] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12301.412444] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12301.425753] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12301.442050] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12301.451458] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[12301.463718] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[12301.470685] RAX: ffff8800b2a37468 RBX: 0000000000000001 RCX: 0000000000000000
[12301.479676] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12301.488658] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12301.497625] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12301.506578] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[12301.515532] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12301.525562] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12301.532972] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12301.541938] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12301.550902] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12301.559851] Stack:
[12301.563072] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12301.572358] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12301.581633] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12301.590935] Call Trace:
[12301.594661] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12301.603830] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12301.612714] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12301.621387] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12301.628596] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12301.637954] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12301.644387] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12301.652668] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12301.659682] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12301.667955] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[12329.271972] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12329.281647] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12329.413521] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12329.426829] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12329.443128] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12329.452534] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[12329.464794] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[12329.471758] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[12329.480750] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12329.489741] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12329.498706] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12329.507666] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12329.516618] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12329.526646] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12329.534054] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12329.543032] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12329.551996] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12329.560945] Stack:
[12329.564158] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12329.573439] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12329.582734] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12329.592026] Call Trace:
[12329.595758] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12329.604922] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12329.613796] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12329.622472] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12329.629688] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12329.639044] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12329.645480] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12329.653759] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12329.660771] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12329.669042] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[12343.856480] INFO: rcu_sched self-detected stall on CPU { 24} (t=4920102 jiffies g=907112 c=907111 q=11879033)
[12343.869035] Task dump for CPU 24:
[12343.873790] OI_scrub R running task 0 5551 2 0x00000088
[12343.882719] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[12343.892056] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[12343.901382] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[12343.910682] Call Trace:
[12343.914382] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[12343.922377] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[12343.929357] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[12343.936892] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[12343.944606] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[12343.952883] [<ffffffff81099697>] update_process_times+0x47/0x80
[12343.960453] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[12343.968484] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[12343.975611] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[12343.983210] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[12343.990513] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[12343.997392] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[12344.005425] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[12344.013261] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[12344.020685] <EOI> [<ffffffffa121cff0>] ? osd_inode_iteration+0x2a0/0xcc0 [osd_ldiskfs]
[12344.030498] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[12344.039595] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12344.048499] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12344.057110] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12344.065524] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12344.072474] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12344.081562] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12344.087730] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12344.095747] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12344.102497] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12369.273366] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12369.282758] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12369.412406] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12369.425591] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12369.441674] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12369.450969] RIP: 0010:[<ffffffffa121ba6b>] [<ffffffffa121ba6b>] osd_scrub_exec+0xdb/0x5b0 [osd_ldiskfs]
[12369.462530] RSP: 0018:ffff880703603c78 EFLAGS: 00000202
[12369.469387] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[12369.478291] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12369.487181] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12369.496077] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12369.504974] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12369.513868] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12369.523836] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12369.531179] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12369.540086] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12369.548994] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12369.557900] Stack:
[12369.561067] ffff880703603d08 0000000000000001 ffff880703603df0 000000009f8f2001
[12369.570305] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[12369.579569] 0000000020000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[12369.588833] Call Trace:
[12369.592510] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12369.601648] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12369.610781] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12369.619631] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12369.628279] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12369.635460] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12369.644772] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12369.651169] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12369.659420] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12369.666398] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12369.674638] Code: fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 <74> 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00
[12397.274341] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12397.283932] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12397.415627] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12397.428917] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12397.445160] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12397.454541] RIP: 0010:[<ffffffffa1217a4b>] [<ffffffffa1217a4b>] osd_scrub_next+0x2b/0x3c0 [osd_ldiskfs]
[12397.466192] RSP: 0018:ffff880703603c70 EFLAGS: 00000282
[12397.473131] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: ffff880703603d98
[12397.482111] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12397.491073] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[12397.500022] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12397.508971] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12397.517920] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12397.527950] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12397.535357] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12397.544327] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12397.553279] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12397.562226] Stack:
[12397.565437] 000000009f8f2001 ffff880703603d08 000000009f8f2001 ffff880703603df0
[12397.574726] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[12397.584016] 0000000000000001 ffff88042bd78800 ffff880703603d98 ffff880703603d78
[12397.593307] Call Trace:
[12397.597023] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[12397.606180] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12397.615334] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12397.624190] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12397.632843] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12397.640049] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12397.649397] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12397.655820] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12397.664088] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12397.671095] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12397.679353] Code: 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87
[12425.275316] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12425.284956] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12425.416590] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12425.429868] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12425.446102] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12425.455478] RIP: 0010:[<ffffffffa121d1c3>] [<ffffffffa121d1c3>] osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[12425.467709] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[12425.474646] RAX: 0000000000000004 RBX: ffff880703603d98 RCX: ffff880703603d98
[12425.483619] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[12425.492571] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[12425.501516] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[12425.510462] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[12425.519394] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12425.529419] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12425.536811] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12425.545769] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12425.554719] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12425.563654] Stack:
[12425.566850] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12425.576120] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12425.585419] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12425.594709] Call Trace:
[12425.598420] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12425.607577] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12425.616458] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12425.625123] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12425.632323] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12425.641670] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12425.648095] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12425.656368] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12425.663374] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12425.671648] Code: 00 00 48 c7 05 9b 86 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 <4c> 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff
[12453.276293] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12453.285944] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12453.417832] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12453.431135] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12453.447424] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12453.456833] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[12453.468503] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[12453.475457] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[12453.484446] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12453.493427] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12453.502376] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12453.511326] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12453.520271] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12453.530290] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12453.537698] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12453.546662] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12453.555622] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12453.564569] Stack:
[12453.567783] ffff880703603d08 ffff880703603d57 ffff880703603df0 000000009f8f2001
[12453.577072] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[12453.586364] 0000000000000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[12453.595661] Call Trace:
[12453.599388] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12453.608556] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12453.617720] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12453.626594] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12453.635283] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12453.642492] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12453.651839] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12453.658267] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12453.666548] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12453.673559] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12453.681828] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[12481.277268] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12481.286943] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12481.418594] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12481.431881] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12481.448133] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12481.457518] RIP: 0010:[<ffffffffa1217a20>] [<ffffffffa1217a20>] osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12481.469260] RSP: 0018:ffff880703603d10 EFLAGS: 00000287
[12481.476201] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[12481.485179] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12481.494154] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[12481.503111] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[12481.512062] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12481.521016] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12481.531043] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12481.538453] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12481.547418] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12481.556378] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12481.565317] Stack:
[12481.568530] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[12481.577816] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[12481.587104] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[12481.596393] Call Trace:
[12481.600111] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[12481.609471] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12481.618626] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12481.627492] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12481.636155] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12481.643372] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12481.652735] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12481.659166] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12481.667445] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12481.674457] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12481.682727] Code: f7 48 8d 04 80 48 c1 e0 03 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd
[12509.278242] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12509.287999] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12509.419781] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12509.433072] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12509.449331] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12509.458717] RIP: 0010:[<ffffffffa121cfb9>] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[12509.470976] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[12509.477927] RAX: 00000000000072ce RBX: 000000009f8f2001 RCX: ffff88042bd78800
[12509.486910] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[12509.495870] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12509.504835] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[12509.513787] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12509.522749] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12509.532781] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12509.540192] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12509.549164] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12509.558126] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12509.567076] Stack:
[12509.570292] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12509.579599] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12509.588895] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12509.598204] Call Trace:
[12509.601929] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12509.611098] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12509.619976] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12509.628649] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12509.635859] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12509.645198] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12509.651630] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12509.659915] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12509.666931] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12509.675212] Code: 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff <42> 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83
[12523.865751] INFO: rcu_sched self-detected stall on CPU { 24} (t=5100105 jiffies g=907112 c=907111 q=13351514)
[12523.878243] Task dump for CPU 24:
[12523.882999] OI_scrub R running task 0 5551 2 0x00000088
[12523.891935] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[12523.901272] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[12523.910592] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[12523.919898] Call Trace:
[12523.923604] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[12523.931603] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[12523.938580] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[12523.946114] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[12523.953822] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[12523.962101] [<ffffffff81099697>] update_process_times+0x47/0x80
[12523.969667] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[12523.977699] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[12523.984846] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[12523.992449] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[12523.999747] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[12524.006630] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[12524.014667] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[12524.022496] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[12524.029923] <EOI> [<ffffffffa121d1f0>] ? osd_inode_iteration+0x4a0/0xcc0 [osd_ldiskfs]
[12524.039746] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12524.048850] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12524.057758] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12524.066383] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12524.074797] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12524.081760] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12524.090848] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12524.097022] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12524.105032] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12524.111793] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12549.279635] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12549.289106] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12549.418758] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12549.431934] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12549.448014] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12549.457313] RIP: 0010:[<ffffffffa121b9c4>] [<ffffffffa121b9c4>] osd_scrub_exec+0x34/0x5b0 [osd_ldiskfs]
[12549.468877] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[12549.475738] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[12549.484649] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12549.493549] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12549.502448] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12549.511348] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12549.520242] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12549.530209] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12549.537560] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12549.546466] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12549.555373] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12549.564280] Stack:
[12549.567450] 0000000000000004 ffff880703603d57 ffff880703603df0 0000000000000000
[12549.576720] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[12549.585977] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff880419dc3a00
[12549.595242] Call Trace:
[12549.598946] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12549.608087] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12549.617232] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12549.626075] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12549.634746] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12549.641933] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12549.651250] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12549.657645] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12549.665903] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12549.672891] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12549.681135] Code: 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48 89 45 d0 <31> c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04 0f 84 84 00
[12577.280611] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12577.290217] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12577.421899] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12577.435188] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12577.451440] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12577.460825] RIP: 0010:[<ffffffffa121cfbd>] [<ffffffffa121cfbd>] osd_inode_iteration+0x26d/0xcc0 [osd_ldiskfs]
[12577.473065] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[12577.480009] RAX: 00000000000072ce RBX: 000000009f8f2001 RCX: ffff88042bd78800
[12577.488997] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[12577.497963] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12577.506921] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12577.515876] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12577.524824] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12577.534868] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12577.542295] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12577.551267] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12577.560229] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12577.569172] Stack:
[12577.572389] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12577.581679] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12577.590974] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12577.600267] Call Trace:
[12577.603985] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12577.613147] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12577.622023] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12577.630693] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12577.637897] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12577.647249] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12577.653668] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12577.661952] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12577.668963] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12577.677233] Code: f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 <48> 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00
[12605.281586] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12605.291246] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12605.423158] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12605.436467] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12605.452748] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12605.462155] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[12605.473822] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[12605.480785] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[12605.489762] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12605.498735] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12605.507692] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12605.516649] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12605.525611] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12605.535643] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12605.543057] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12605.552030] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12605.560998] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12605.569947] Stack:
[12605.573159] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 ffff880703603d57
[12605.582446] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[12605.591716] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[12605.600992] Call Trace:
[12605.604718] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12605.613876] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12605.623028] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12605.631892] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12605.640563] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12605.647762] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12605.657099] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12605.663523] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12605.671792] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12605.678792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12605.687051] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[12633.282562] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12633.292237] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12633.423764] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12633.437043] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12633.453282] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12633.462658] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[12633.474903] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[12633.481828] RAX: 0000000020000000 RBX: ffff880703603d98 RCX: 0000000000000000
[12633.490801] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12633.499753] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12633.508700] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[12633.517632] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[12633.526578] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12633.536594] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12633.543989] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12633.552935] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12633.561877] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12633.570807] Stack:
[12633.574011] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12633.583290] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12633.592580] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12633.601851] Call Trace:
[12633.605552] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12633.614701] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12633.623566] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12633.632236] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12633.639443] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12633.648784] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12633.655212] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12633.663498] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12633.670505] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12633.678779] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[12661.283537] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12661.293223] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12661.425131] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12661.438431] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12661.454723] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12661.464126] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[12661.476353] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[12661.483309] RAX: ffff8800b2a37468 RBX: ffff880703603d57 RCX: 0000000000000000
[12661.492303] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12661.501285] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12661.510233] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12661.519179] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[12661.528120] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12661.538139] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12661.545537] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12661.554511] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12661.563553] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12661.572499] Stack:
[12661.575708] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12661.585004] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12661.594286] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12661.603575] Call Trace:
[12661.607308] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12661.616472] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12661.625344] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12661.634020] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12661.641240] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12661.650600] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12661.657035] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12661.665326] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12661.672336] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12661.680605] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[12689.284512] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12689.294162] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12689.426076] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12689.439391] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12689.455681] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12689.465097] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[12689.477362] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[12689.484327] RAX: ffff8800b2a37468 RBX: ffff880703603d98 RCX: 0000000000000000
[12689.493318] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12689.502297] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12689.511269] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[12689.520212] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[12689.529156] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12689.539174] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12689.546580] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12689.555546] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12689.564499] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12689.573444] Stack:
[12689.576647] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12689.585918] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12689.595201] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12689.604497] Call Trace:
[12689.608212] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12689.617367] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12689.626230] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12689.634905] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12689.642111] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12689.651455] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12689.657877] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12689.666146] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12689.673148] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12689.681410] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[12703.875030] INFO: rcu_sched self-detected stall on CPU { 24} (t=5280108 jiffies g=907112 c=907111 q=14494531)
[12703.887510] Task dump for CPU 24:
[12703.892253] OI_scrub R running task 0 5551 2 0x00000088
[12703.901175] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[12703.910512] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[12703.919832] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[12703.929123] Call Trace:
[12703.932815] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[12703.940816] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[12703.947781] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[12703.955307] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[12703.963013] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[12703.971274] [<ffffffff81099697>] update_process_times+0x47/0x80
[12703.978833] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[12703.986856] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[12703.993982] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[12704.001575] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[12704.008861] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[12704.015736] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[12704.023779] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[12704.031600] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[12704.039015] <EOI> [<ffffffffa1217d70>] ? osd_scrub_next+0x350/0x3c0 [osd_ldiskfs]
[12704.048335] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[12704.057228] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12704.066119] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12704.074720] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12704.083135] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12704.090088] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12704.099171] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12704.105343] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12704.113363] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12704.120119] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12719.514071] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[12719.528038] Lustre: Skipped 1 previous similar message
[12719.535516] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[12719.549633] Lustre: Skipped 2 previous similar messages
[12729.285906] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12729.295372] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12729.425421] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12729.438612] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12729.454695] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12729.464008] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[12729.475608] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[12729.482513] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[12729.491460] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12729.500403] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12729.509342] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12729.518288] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12729.527243] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12729.537268] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12729.544670] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12729.553640] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12729.562613] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12729.571570] Stack:
[12729.574799] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[12729.584113] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[12729.593437] 0000000020000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[12729.602730] Call Trace:
[12729.606484] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12729.615682] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12729.624855] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12729.633748] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12729.642437] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12729.649675] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12729.659057] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12729.665516] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12729.673814] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12729.680845] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12729.689137] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[12757.286890] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12757.296557] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12757.428393] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12757.441679] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12757.457938] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12757.467325] RIP: 0010:[<ffffffffa121cfda>] [<ffffffffa121cfda>] osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[12757.479563] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[12757.486502] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[12757.495480] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[12757.504445] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[12757.513417] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12757.522378] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12757.531338] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12757.541375] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12757.548792] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12757.557774] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12757.566739] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12757.575691] Stack:
[12757.578901] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12757.588212] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12757.597512] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12757.606825] Call Trace:
[12757.610558] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12757.619733] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12757.628605] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12757.637285] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12757.644500] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12757.653860] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12757.660300] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12757.668588] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12757.675610] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12757.683891] Code: a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff <48> 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[12785.287856] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12785.297507] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12785.429694] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12785.443006] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12785.459313] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12785.468721] RIP: 0010:[<ffffffffa1217b69>] [<ffffffffa1217b69>] osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[12785.480494] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[12785.487458] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[12785.496448] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12785.505444] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[12785.514424] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12785.523380] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12785.532334] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12785.542373] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12785.549771] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12785.558737] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12785.567705] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12785.576655] Stack:
[12785.579869] 000000009f8f2001 ffff880703603d08 000000009f8f2001 ffff880703603df0
[12785.589157] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[12785.598432] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[12785.607731] Call Trace:
[12785.611459] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[12785.620619] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12785.629785] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12785.638657] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12785.647320] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12785.654520] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12785.663863] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12785.670285] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12785.678560] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12785.685563] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12785.693825] Code: 45 90 74 49 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 <48> 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83
[12813.288831] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[12813.298478] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12813.430275] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12813.443548] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12813.459775] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12813.469162] RIP: 0010:[<ffffffffa121d1fc>] [<ffffffffa121d1fc>] osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[12813.481399] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[12813.488338] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[12813.497322] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12813.506290] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12813.515239] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12813.524182] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffff880703603d08
[12813.533134] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12813.543159] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12813.550560] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12813.559527] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12813.568488] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12813.577440] Stack:
[12813.580658] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12813.589956] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12813.599245] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12813.608541] Call Trace:
[12813.612259] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12813.621426] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12813.630305] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12813.638977] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12813.646185] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12813.655544] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12813.661980] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12813.670270] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12813.677287] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12813.685571] Code: 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 <48> 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2
[12841.289821] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12841.299509] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12841.431746] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12841.445057] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12841.461354] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12841.470758] RIP: 0010:[<ffffffffa121b9cd>] [<ffffffffa121b9cd>] osd_scrub_exec+0x3d/0x5b0 [osd_ldiskfs]
[12841.482429] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[12841.489397] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[12841.498393] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12841.507381] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12841.516348] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12841.525307] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12841.534257] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12841.544282] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12841.551687] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12841.560652] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12841.569617] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12841.578567] Stack:
[12841.581796] 0000000000000004 0000000000000001 ffffffffa1217a20 ffff880703603d98
[12841.591080] ffff880703603d57 ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10
[12841.600387] ffffffffa1217a3b 0000000000000010 0000000000000287 ffff880703603ce0
[12841.609676] Call Trace:
[12841.613407] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12841.622280] [<ffffffffa1217a3b>] ? osd_scrub_next+0x1b/0x3c0 [osd_ldiskfs]
[12841.631058] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12841.640222] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12841.649385] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12841.658248] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12841.666917] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12841.674120] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12841.683472] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12841.689903] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12841.698173] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12841.705185] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12841.713459] Code: 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8d 43 08 48 85 db <4c> 0f 45 e0 41 83 f9 04 0f 84 84 00 00 00 41 83 f9 07 74 4f 41
[12869.290789] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12869.300417] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12869.432078] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12869.445342] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12869.461545] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12869.470902] RIP: 0010:[<ffffffffa121ba25>] [<ffffffffa121ba25>] osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[12869.482531] RSP: 0018:ffff880703603d08 EFLAGS: 00000292
[12869.489450] RAX: 0000000000000000 RBX: ffff880703603d57 RCX: 0000000000000000
[12869.498414] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12869.507365] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12869.516324] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12869.525282] R13: ffffea00027e3c80 R14: ffff880703603c70 R15: ffff880703603d08
[12869.534242] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12869.544278] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12869.551685] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12869.560647] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12869.569596] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12869.578546] Stack:
[12869.581779] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[12869.591092] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[12869.600379] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[12869.609656] Call Trace:
[12869.613376] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12869.622540] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12869.631688] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12869.640543] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12869.649222] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12869.656418] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12869.665769] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12869.672207] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12869.680491] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12869.687506] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12869.695792] Code: 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d
[12883.884298] INFO: rcu_sched self-detected stall on CPU { 24} (t=5460111 jiffies g=907112 c=907111 q=15277119)
[12883.896715] Task dump for CPU 24:
[12883.901434] OI_scrub R running task 0 5551 2 0x00000088
[12883.910341] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[12883.919661] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[12883.928958] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[12883.938250] Call Trace:
[12883.941932] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[12883.949919] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[12883.956871] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[12883.964384] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[12883.972074] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[12883.980327] [<ffffffff81099697>] update_process_times+0x47/0x80
[12883.987880] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[12883.995890] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[12884.003012] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[12884.010600] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[12884.017871] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[12884.024745] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[12884.032775] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[12884.040591] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[12884.048021] <EOI> [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12884.057658] [<ffffffffa121b9a2>] ? osd_scrub_exec+0x12/0x5b0 [osd_ldiskfs]
[12884.066172] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12884.075083] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12884.083985] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12884.092602] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12884.101007] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12884.107956] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12884.117038] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12884.123218] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12884.131232] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12884.137993] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12909.292184] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12909.301577] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12909.431348] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12909.444530] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12909.460567] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12909.469846] RIP: 0010:[<ffffffffa121d00e>] [<ffffffffa121d00e>] osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[12909.481987] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[12909.488853] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[12909.497749] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[12909.506648] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[12909.515550] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[12909.524439] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[12909.533323] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12909.543287] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12909.550631] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12909.559537] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12909.568436] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12909.577333] Stack:
[12909.580509] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12909.589757] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12909.598993] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12909.608258] Call Trace:
[12909.611943] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12909.621071] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12909.629908] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12909.638550] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12909.645735] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12909.655059] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12909.661451] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12909.669714] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12909.676697] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12909.684943] Code: 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 <66> 90 0f 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90
[12937.293158] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12937.302797] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12937.434998] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12937.448317] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12937.464610] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12937.474024] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[12937.486273] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[12937.493230] RAX: 0000000020000000 RBX: 0000000000000000 RCX: 0000000000000000
[12937.502232] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12937.511218] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[12937.520177] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[12937.529132] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[12937.538085] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12937.548112] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12937.555515] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12937.564487] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12937.573455] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12937.582402] Stack:
[12937.585621] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[12937.594901] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[12937.604180] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[12937.613473] Call Trace:
[12937.617217] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12937.626381] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12937.635261] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12937.643953] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12937.651166] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12937.660534] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12937.666971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12937.675259] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12937.682278] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12937.690549] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[12965.294134] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12965.303818] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12965.435850] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12965.449156] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12965.465443] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12965.474835] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[12965.486497] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[12965.493454] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[12965.502434] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12965.511408] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12965.520365] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[12965.529312] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[12965.538267] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12965.548293] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12965.555695] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12965.564671] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12965.573636] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12965.582589] Stack:
[12965.585807] ffff880703603df0 0000000000000001 ffff880703603df0 000000009f8f2001
[12965.595108] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[12965.604400] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[12965.613699] Call Trace:
[12965.617428] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12965.626583] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12965.635739] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12965.644615] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12965.653290] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12965.660498] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12965.669856] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12965.676286] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12965.684567] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12965.691578] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12965.699834] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[12993.295109] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[12993.304759] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[12993.436954] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[12993.450243] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[12993.466504] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[12993.475891] RIP: 0010:[<ffffffffa121ba25>] [<ffffffffa121ba25>] osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[12993.487546] RSP: 0018:ffff880703603d08 EFLAGS: 00000292
[12993.494488] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000
[12993.503465] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[12993.512438] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[12993.521399] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[12993.530350] R13: ffffea00027e3c80 R14: ffff880703603c78 R15: ffff880703603d08
[12993.539299] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[12993.549327] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[12993.556735] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[12993.565709] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[12993.574664] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[12993.583607] Stack:
[12993.586823] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[12993.596094] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[12993.605381] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[12993.614662] Call Trace:
[12993.618389] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[12993.627553] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[12993.636718] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[12993.645588] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[12993.654264] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[12993.661475] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[12993.670831] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[12993.677270] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12993.685552] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[12993.692566] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[12993.700840] Code: 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d
[13021.296077] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13021.305734] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13021.437866] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13021.451159] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13021.467436] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13021.476833] RIP: 0010:[<ffffffffa1217b81>] [<ffffffffa1217b81>] osd_scrub_next+0x161/0x3c0 [osd_ldiskfs]
[13021.488603] RSP: 0018:ffff880703603ce8 EFLAGS: 00000282
[13021.495547] RAX: 0000000000000004 RBX: ffff880703603c78 RCX: ffff880703603d98
[13021.504535] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[13021.513498] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13021.522455] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[13021.531406] R13: ffffffffa121b9c4 R14: ffffffffffffff10 R15: ffff8803162f6000
[13021.540358] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13021.550390] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13021.557795] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13021.566757] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13021.575717] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13021.584662] Stack:
[13021.587875] ffff8800b2a3746c 0000000000000000 00000000ffffffff 0000000000000001
[13021.597176] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[13021.606474] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[13021.615770] Call Trace:
[13021.619499] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13021.628668] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13021.637832] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13021.646700] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13021.655367] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13021.662580] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13021.671937] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13021.678381] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13021.686671] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13021.693687] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13021.701959] Code: 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b <41> 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0 0f
[13049.297053] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13049.306735] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13049.438732] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13049.452020] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13049.468293] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13049.477697] RIP: 0010:[<ffffffffa121b9b3>] [<ffffffffa121b9b3>] osd_scrub_exec+0x23/0x5b0 [osd_ldiskfs]
[13049.489356] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[13049.496302] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[13049.505292] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13049.514252] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13049.523208] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13049.532158] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13049.541111] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13049.551150] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13049.558567] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13049.567542] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13049.576513] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13049.585465] Stack:
[13049.588682] ffff880703603d08 000000009f8f2001 ffff880703603df0 000000009f8f2001
[13049.597977] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[13049.607267] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[13049.616565] Call Trace:
[13049.620287] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13049.629454] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13049.638615] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13049.647478] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13049.656140] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13049.663349] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13049.672707] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13049.679137] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13049.687414] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13049.694426] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13049.702694] Code: 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 <4c> 89 45 90 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8d
[13063.893562] INFO: rcu_sched self-detected stall on CPU { 24} (t=5640114 jiffies g=907112 c=907111 q=15703893)
[13063.906080] Task dump for CPU 24:
[13063.910821] OI_scrub R running task 0 5551 2 0x00000088
[13063.919745] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[13063.929067] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[13063.938370] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[13063.947636] Call Trace:
[13063.951316] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[13063.959297] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[13063.966254] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[13063.973779] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[13063.981468] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[13063.989729] [<ffffffff81099697>] update_process_times+0x47/0x80
[13063.997282] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[13064.005307] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[13064.012431] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[13064.020014] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[13064.027281] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[13064.034140] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[13064.042160] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[13064.049989] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[13064.057418] <EOI> [<ffffffffa121ba87>] ? osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[13064.066652] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13064.075549] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13064.084440] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13064.093049] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13064.101450] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13064.108401] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13064.117487] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13064.123647] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13064.131658] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13064.138398] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13089.298448] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13089.307873] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13089.437773] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13089.450950] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13089.467018] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13089.476307] RIP: 0010:[<ffffffffa1217b87>] [<ffffffffa1217b87>] osd_scrub_next+0x167/0x3c0 [osd_ldiskfs]
[13089.487974] RSP: 0018:ffff880703603d00 EFLAGS: 00000282
[13089.494827] RAX: 0000000000000004 RBX: 0000000000000001 RCX: ffff880703603d98
[13089.503738] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[13089.512627] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13089.521520] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13089.530410] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[13089.539296] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13089.549249] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13089.556574] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13089.565472] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13089.574369] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13089.583248] Stack:
[13089.586409] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[13089.595641] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[13089.604867] ffff8800b2a37468 0000000020000000 010000000000000c 0000000000000000
[13089.614094] Call Trace:
[13089.617753] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13089.626872] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13089.635987] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13089.644811] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13089.653455] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13089.660627] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13089.669931] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13089.676326] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13089.684576] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13089.691550] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13089.699791] Code: 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41 5c 41 5d 41 5e <41> 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0 0f 84 c2 fe ff ff 0f
[13117.299422] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13117.309070] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13117.440718] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13117.453993] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13117.470224] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13117.479602] RIP: 0010:[<ffffffffa1217b80>] [<ffffffffa1217b80>] osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[13117.491347] RSP: 0018:ffff880703603ce0 EFLAGS: 00000282
[13117.498275] RAX: 0000000000000004 RBX: ffff880703603c78 RCX: ffff880703603d98
[13117.507240] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[13117.516201] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13117.525138] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[13117.534073] R13: ffffffffa121ba87 R14: ffffffffffffff10 R15: ffff8803162f6000
[13117.543009] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13117.553027] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13117.560434] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13117.569393] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13117.578351] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13117.587305] Stack:
[13117.590518] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[13117.599816] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[13117.609117] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[13117.618412] Call Trace:
[13117.622132] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13117.631297] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13117.640458] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13117.649321] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13117.657983] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13117.665198] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13117.674550] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13117.680977] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13117.689283] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13117.696299] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13117.704574] Code: 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0
[13145.300397] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13145.310074] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13145.441818] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13145.455109] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13145.471374] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13145.480764] RIP: 0010:[<ffffffffa1162b36>] [<ffffffffa1162b36>] ldiskfs_itable_unused_count+0x6/0x30 [ldiskfs]
[13145.493112] RSP: 0018:ffff880703603d08 EFLAGS: 00000286
[13145.500064] RAX: ffff8800b2a37468 RBX: 0000000000000001 RCX: 0000000000000000
[13145.509044] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[13145.518030] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13145.526985] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13145.535927] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000282
[13145.544873] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13145.554899] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13145.562310] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13145.571278] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13145.580239] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13145.589188] Stack:
[13145.592407] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[13145.601701] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[13145.610999] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[13145.620295] Call Trace:
[13145.624031] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[13145.633389] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13145.642548] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13145.651418] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13145.660079] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13145.667287] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13145.676640] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13145.683076] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13145.691359] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13145.698368] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13145.706642] Code: 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 <48> 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76
[13173.301372] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13173.311009] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13173.442884] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13173.456182] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13173.472446] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13173.481838] RIP: 0010:[<ffffffffa121ba64>] [<ffffffffa121ba64>] osd_scrub_exec+0xd4/0x5b0 [osd_ldiskfs]
[13173.493500] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[13173.500439] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[13173.509423] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13173.518386] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13173.527335] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13173.536281] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13173.545232] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13173.555265] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13173.562670] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13173.571638] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13173.580597] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13173.589546] Stack:
[13173.592760] ffff880703603df0 ffff880703603d57 ffff880703603df0 000000009f8f2001
[13173.602054] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[13173.611362] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[13173.620650] Call Trace:
[13173.624373] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13173.633555] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13173.642717] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13173.651582] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13173.660244] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13173.667452] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13173.676804] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13173.683232] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13173.691523] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13173.698532] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13173.706798] Code: 4d 88 4c 89 ef e8 1d fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 <f6> 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00
[13201.302348] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13201.312026] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13201.443757] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13201.457044] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13201.473293] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13201.482676] RIP: 0010:[<ffffffffa121b9cd>] [<ffffffffa121b9cd>] osd_scrub_exec+0x3d/0x5b0 [osd_ldiskfs]
[13201.494328] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[13201.501266] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[13201.510251] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13201.519220] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13201.528168] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13201.537121] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13201.546076] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13201.556103] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13201.563508] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13201.572475] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13201.581441] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13201.590382] Stack:
[13201.593589] ffff880703603df0 000000009f8f2001 ffff880703603df0 000000009f8f2001
[13201.602887] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[13201.612170] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[13201.621464] Call Trace:
[13201.625198] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13201.634358] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13201.643514] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13201.652379] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13201.661040] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13201.668247] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13201.677587] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13201.684017] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13201.692293] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13201.699304] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13201.707564] Code: 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8d 43 08 48 85 db <4c> 0f 45 e0 41 83 f9 04 0f 84 84 00 00 00 41 83 f9 07 74 4f 41
[13229.303323] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13229.312997] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13229.444770] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13229.458062] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13229.474300] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13229.483691] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[13229.495340] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[13229.502273] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[13229.511244] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13229.520201] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13229.529152] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13229.538094] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13229.547037] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13229.557060] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13229.564460] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13229.573431] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13229.582378] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13229.591321] Stack:
[13229.594525] ffff880703603d08 0000000000000001 ffffea00027e3c80 000000009f8f2001
[13229.603808] ffff880703603d57 ffff880703603d57 00000000000072ce ffff88042bd78800
[13229.613103] 00000000000072ce ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[13229.622405] Call Trace:
[13229.626125] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13229.635300] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13229.644459] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13229.653319] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13229.661987] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13229.669194] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13229.678541] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13229.684977] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13229.693274] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13229.700282] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13229.708554] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[13243.902832] INFO: rcu_sched self-detected stall on CPU { 24} (t=5820117 jiffies g=907112 c=907111 q=16299401)
[13243.915297] Task dump for CPU 24:
[13243.920023] OI_scrub R running task 0 5551 2 0x00000088
[13243.928960] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[13243.938277] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[13243.947556] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[13243.956839] Call Trace:
[13243.960524] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[13243.968518] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[13243.975483] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[13243.983007] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[13243.990704] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[13243.998972] [<ffffffff81099697>] update_process_times+0x47/0x80
[13244.006527] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[13244.014540] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[13244.021669] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[13244.029257] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[13244.036545] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[13244.043435] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[13244.051469] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[13244.059287] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[13244.066710] <EOI> [<ffffffffa1217a4b>] ? osd_scrub_next+0x2b/0x3c0 [osd_ldiskfs]
[13244.075956] [<ffffffffa121ba1d>] ? osd_scrub_exec+0x8d/0x5b0 [osd_ldiskfs]
[13244.084471] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13244.093375] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13244.102278] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13244.110881] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13244.119293] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13244.126241] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13244.135341] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13244.141521] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13244.149554] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13244.156305] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13269.304723] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13269.314164] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13269.443867] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13269.457038] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13269.473064] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13269.482341] RIP: 0010:[<ffffffffa121b999>] [<ffffffffa121b999>] osd_scrub_exec+0x9/0x5b0 [osd_ldiskfs]
[13269.493802] RSP: 0018:ffff880703603d08 EFLAGS: 00000293
[13269.500654] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[13269.509547] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13269.518448] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13269.527349] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13269.536242] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000282
[13269.545122] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13269.555073] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13269.562404] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13269.571300] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13269.580193] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13269.589076] Stack:
[13269.592229] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[13269.601479] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[13269.610720] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[13269.619973] Call Trace:
[13269.623651] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13269.632780] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13269.641907] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13269.650739] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13269.659379] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13269.666544] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13269.675845] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13269.682240] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13269.690484] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13269.697462] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13269.705687] Code: 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 <41> 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b
[13297.305699] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13297.315338] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13297.447098] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13297.460372] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13297.476608] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13297.485976] RIP: 0010:[<ffffffffa1217d70>] [<ffffffffa1217d70>] osd_scrub_next+0x350/0x3c0 [osd_ldiskfs]
[13297.497714] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[13297.504641] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[13297.513610] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13297.522569] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13297.531516] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13297.540461] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13297.549403] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13297.559433] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13297.566837] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13297.575801] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13297.584761] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13297.593705] Stack:
[13297.596924] 000000009f8f2001 ffff880703603d08 000000009f8f2001 ffff880703603df0
[13297.606222] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[13297.615527] ffff880703603d57 0000000000000000 0000000000000000 ffff880703603d78
[13297.624827] Call Trace:
[13297.628542] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13297.637702] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13297.646857] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13297.655714] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13297.664369] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13297.671572] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13297.680913] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13297.687336] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13297.695614] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13297.702618] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13297.710882] Code: 97 ff 48 8b 55 90 48 8b 4d 88 44 8b 85 74 ff ff ff e9 1d ff ff ff 48 89 d7 44 89 45 90 e8 f9 67 47 e0 44 8b 45 90 0f 1f 44 00 00 <45> 84 c0 b8 04 00 00 00 0f 85 eb fd ff ff 49 8d b7 6c 14 00 00
[13325.306667] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13325.316359] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13325.448145] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13325.461437] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13325.477678] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13325.487057] RIP: 0010:[<ffffffffa121ba12>] [<ffffffffa121ba12>] osd_scrub_exec+0x82/0x5b0 [osd_ldiskfs]
[13325.498697] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[13325.505633] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[13325.514613] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13325.523580] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13325.532525] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13325.541467] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13325.550410] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13325.560431] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13325.567831] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13325.576789] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13325.585735] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13325.594670] Stack:
[13325.597870] ffff880703603d08 000000009f8f2001 ffff880703603df0 000000009f8f2001
[13325.607138] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[13325.616433] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[13325.625721] Call Trace:
[13325.629427] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13325.638598] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13325.647751] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13325.656605] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13325.665259] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13325.672460] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13325.681808] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13325.688231] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13325.696501] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13325.703504] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13325.711769] Code: 48 89 ca 44 89 c9 e8 0e c6 ff ff 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 <0f> 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f 5d c3
[13353.307643] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13353.317359] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13353.449125] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13353.462409] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13353.478654] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13353.488023] RIP: 0010:[<ffffffffa121cfb9>] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[13353.500248] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[13353.507185] RAX: 00000000000072ce RBX: 000000009f8f2001 RCX: ffff88042bd78800
[13353.516165] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[13353.525121] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13353.534068] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13353.543012] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[13353.551961] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13353.561982] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13353.569387] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13353.578345] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13353.587290] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13353.596229] Stack:
[13353.599444] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13353.608725] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13353.618025] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13353.627317] Call Trace:
[13353.631039] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13353.640209] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13353.649077] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13353.657751] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13353.664955] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13353.674309] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13353.680740] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13353.689024] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13353.696036] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13353.704314] Code: 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff <42> 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83
[13373.553202] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[13373.566529] Lustre: Skipped 1 previous similar message
[13373.573360] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[13373.586773] Lustre: Skipped 1 previous similar message
[13381.308618] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13381.318309] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13381.450048] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13381.463337] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13381.479591] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13381.488987] RIP: 0010:[<ffffffffa1217a3b>] [<ffffffffa1217a3b>] osd_scrub_next+0x1b/0x3c0 [osd_ldiskfs]
[13381.500653] RSP: 0018:ffff880703603ce0 EFLAGS: 00000287
[13381.507616] RAX: ffffffffa1217a20 RBX: ffff880703603c78 RCX: ffff880703603d98
[13381.516619] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13381.525627] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13381.534637] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[13381.543644] R13: ffffffffa121ba64 R14: ffffffffffffff10 R15: ffff8803162f6000
[13381.552650] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13381.562723] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13381.570173] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13381.579182] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13381.588185] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13381.597192] Stack:
[13381.600451] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[13381.609794] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[13381.619138] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[13381.628497] Call Trace:
[13381.632283] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13381.641496] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13381.650715] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13381.659633] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13381.668363] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13381.675625] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13381.685014] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13381.691495] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13381.699827] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13381.706892] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13381.715210] Code: 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 <48> 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31
[13409.309601] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13409.319277] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13409.451128] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13409.464420] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13409.480677] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13409.490066] RIP: 0010:[<ffffffffa1217a3e>] [<ffffffffa1217a3e>] osd_scrub_next+0x1e/0x3c0 [osd_ldiskfs]
[13409.501719] RSP: 0018:ffff880703603ce0 EFLAGS: 00000287
[13409.508667] RAX: ffffffffa1217a20 RBX: ffff880703603c70 RCX: ffff880703603d98
[13409.517649] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13409.526611] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13409.535568] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[13409.544512] R13: ffffffffa1217a4b R14: ffffffffffffff10 R15: ffff8803162f6000
[13409.553468] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13409.563502] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13409.570913] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13409.579877] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13409.588823] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13409.597757] Stack:
[13409.600968] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[13409.610251] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[13409.619561] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[13409.628878] Call Trace:
[13409.632610] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13409.641781] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13409.650947] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13409.659800] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13409.668472] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13409.675679] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13409.685033] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13409.691470] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13409.699748] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13409.706765] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13409.715034] Code: 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb <48> 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b
[13423.912103] INFO: rcu_sched self-detected stall on CPU { 24} (t=6000120 jiffies g=907112 c=907111 q=17487651)
[13423.924595] Task dump for CPU 24:
[13423.929321] OI_scrub R running task 0 5551 2 0x00000088
[13423.938250] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[13423.947574] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[13423.956874] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[13423.966156] Call Trace:
[13423.969836] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[13423.977809] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[13423.984763] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[13423.992281] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[13423.999970] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[13424.008224] [<ffffffff81099697>] update_process_times+0x47/0x80
[13424.015781] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[13424.023795] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[13424.030918] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[13424.038508] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[13424.045790] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[13424.052672] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[13424.060697] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[13424.068510] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[13424.075940] <EOI> [<ffffffffa121ba64>] ? osd_scrub_exec+0xd4/0x5b0 [osd_ldiskfs]
[13424.085186] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13424.094091] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13424.102998] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13424.111613] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13424.120015] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13424.126959] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13424.136044] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13424.142211] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13424.150232] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13424.156982] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13449.310988] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13449.320402] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13449.450001] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13449.463183] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13449.479246] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13449.488543] RIP: 0010:[<ffffffffa121d1c3>] [<ffffffffa121d1c3>] osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[13449.500690] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[13449.507546] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[13449.516456] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[13449.525347] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[13449.534235] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[13449.543133] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffffea00027e3c80
[13449.552018] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13449.561984] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13449.569325] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13449.578222] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13449.587118] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13449.596012] Stack:
[13449.599171] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13449.608409] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13449.617663] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13449.626919] Call Trace:
[13449.630607] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13449.639726] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13449.648567] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13449.657220] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13449.664406] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13449.673727] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13449.680122] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13449.688375] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13449.695355] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13449.703596] Code: 00 00 48 c7 05 9b 86 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 <4c> 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff
[13459.556421] Lustre: 3694:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1494366804/real 1494366804] req@ffff88028d932d00 x1566943502434336/t0(0) o104->soaked-MDT0000@192.168.1.118@o2ib100:15/16 lens 296/224 e 0 to 1 dl 1494366811 ref 1 fl Rpc:X/0/ffffffff rc 0/-1
[13477.311965] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13477.321663] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13477.453686] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13477.466998] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13477.483303] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13477.492717] RIP: 0010:[<ffffffffa121d1ca>] [<ffffffffa121d1ca>] osd_inode_iteration+0x47a/0xcc0 [osd_ldiskfs]
[13477.504990] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[13477.511972] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[13477.520990] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[13477.530001] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13477.539010] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13477.548030] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffff880703603d08
[13477.557045] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13477.567136] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13477.574603] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13477.583621] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13477.592631] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13477.601636] Stack:
[13477.604924] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13477.614278] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13477.623625] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13477.632970] Call Trace:
[13477.636758] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13477.645984] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13477.654905] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13477.663634] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13477.670901] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13477.680312] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13477.686802] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13477.695136] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13477.702213] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13477.710536] Code: 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff <48> 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[13505.312939] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13505.322653] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13505.454675] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13505.467986] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13505.484289] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13505.493698] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[13505.505375] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[13505.512337] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[13505.521326] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13505.530305] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13505.539278] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13505.548234] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13505.557194] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13505.567223] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13505.574632] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13505.583602] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13505.592567] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13505.601516] Stack:
[13505.604735] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[13505.614007] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[13505.623299] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[13505.632599] Call Trace:
[13505.636320] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13505.645482] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13505.654646] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13505.663511] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13505.672174] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13505.679375] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13505.688714] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13505.695138] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13505.703418] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13505.710421] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13505.718688] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[13533.313914] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13533.323577] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13533.455299] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13533.468581] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13533.484813] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13533.494192] RIP: 0010:[<ffffffffa121d1ca>] [<ffffffffa121d1ca>] osd_inode_iteration+0x47a/0xcc0 [osd_ldiskfs]
[13533.506417] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[13533.513354] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[13533.522329] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[13533.531291] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13533.540240] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13533.549185] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffff880703603d08
[13533.558132] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13533.568159] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13533.575563] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13533.584529] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13533.593473] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13533.602413] Stack:
[13533.605617] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13533.614903] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13533.624223] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13533.633503] Call Trace:
[13533.637217] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13533.646390] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13533.655264] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13533.663936] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13533.671137] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13533.680483] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13533.686917] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13533.695198] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13533.702223] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13533.710489] Code: 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff <48> 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[13561.314889] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13561.324557] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13561.456559] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13561.469867] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13561.486154] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13561.495562] RIP: 0010:[<ffffffffa1162b30>] [<ffffffffa1162b30>] ldiskfs_used_dirs_count+0x30/0x30 [ldiskfs]
[13561.507627] RSP: 0018:ffff880703603d10 EFLAGS: 00000286
[13561.514592] RAX: ffff8800b2a37468 RBX: ffff880703603d57 RCX: 0000000000000000
[13561.523583] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[13561.532565] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13561.541533] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13561.550490] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[13561.559440] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13561.569476] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13561.576882] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13561.585849] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13561.594809] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13561.603755] Stack:
[13561.606963] ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[13561.616261] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[13561.625556] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[13561.634858] Call Trace:
[13561.638603] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[13561.647967] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13561.657133] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13561.666009] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13561.674685] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13561.681894] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13561.691236] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13561.697677] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13561.705958] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13561.712970] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13561.721246] Code: 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 <66> 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89
[13589.315865] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13589.325550] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13589.457328] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13589.470619] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13589.486883] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13589.496272] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[13589.507922] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[13589.514865] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[13589.523847] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13589.532819] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13589.541774] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13589.550720] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13589.559665] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13589.569700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13589.577122] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13589.586098] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13589.595060] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13589.604010] Stack:
[13589.607227] 000000009f8f2001 ffff880703603d08 ffff880703603d57 ffffea00027e3c80
[13589.616522] 000000009f8f2001 ffff880703603d57 ffff880703603d57 ffffffffa121b990
[13589.625834] 0000000000000000 ffff880703603d78 ffff8800b2a36000 ffff8803162f6000
[13589.635126] Call Trace:
[13589.638846] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13589.648005] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13589.657164] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13589.666308] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13589.675152] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13589.683802] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13589.690996] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13589.700327] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13589.706741] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13589.715014] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13589.722005] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13589.730259] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[13603.921386] INFO: rcu_sched self-detected stall on CPU { 24} (t=6180123 jiffies g=907112 c=907111 q=19283214)
[13603.933835] Task dump for CPU 24:
[13603.938544] OI_scrub R running task 0 5551 2 0x00000088
[13603.947461] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[13603.956762] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[13603.966040] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[13603.975285] Call Trace:
[13603.978940] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[13603.986898] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[13603.993828] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[13604.001319] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[13604.008982] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[13604.017206] [<ffffffff81099697>] update_process_times+0x47/0x80
[13604.024726] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[13604.032713] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[13604.039806] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[13604.047363] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[13604.054626] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[13604.061486] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[13604.069508] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[13604.077329] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[13604.084745] <EOI> [<ffffffffa121d1d9>] ? osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[13604.094561] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13604.103652] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13604.112549] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13604.121144] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13604.129537] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13604.136479] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13604.145568] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13604.151736] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13604.159754] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13604.166495] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13629.317264] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13629.326675] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13629.456991] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13629.470172] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13629.486251] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13629.495559] RIP: 0010:[<ffffffffa121d1fc>] [<ffffffffa121d1fc>] osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[13629.507710] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[13629.514579] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[13629.523495] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13629.532404] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13629.541317] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[13629.550228] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[13629.559136] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13629.569123] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13629.576481] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13629.585403] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13629.594332] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13629.603249] Stack:
[13629.606431] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13629.615694] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13629.624949] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13629.634214] Call Trace:
[13629.637905] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13629.647054] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13629.655903] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13629.664572] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13629.671779] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13629.681120] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13629.687533] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13629.695799] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13629.702792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13629.711048] Code: 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 <48> 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2
[13657.318239] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13657.327886] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13657.459789] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13657.473106] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13657.489431] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13657.498837] RIP: 0010:[<ffffffffa121d1f0>] [<ffffffffa121d1f0>] osd_inode_iteration+0x4a0/0xcc0 [osd_ldiskfs]
[13657.511108] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[13657.518081] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[13657.527080] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13657.536063] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13657.545038] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13657.554001] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[13657.562961] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13657.572996] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13657.580411] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13657.589391] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13657.598363] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13657.607322] Stack:
[13657.610542] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13657.619836] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13657.629124] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13657.638432] Call Trace:
[13657.642158] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13657.651336] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13657.660214] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13657.668892] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13657.676112] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13657.685462] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13657.691896] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13657.700180] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13657.707203] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13657.715486] Code: 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c <8b> 85 58 ff ff ff 41 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b
[13685.319213] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13685.328834] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13685.460808] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13685.474115] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13685.490398] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13685.499813] RIP: 0010:[<ffffffffa121ba25>] [<ffffffffa121ba25>] osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[13685.511494] RSP: 0018:ffff880703603d08 EFLAGS: 00000292
[13685.518460] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000
[13685.527448] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13685.536430] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13685.545389] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13685.554348] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[13685.563294] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13685.573306] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13685.580707] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13685.589667] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13685.598625] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13685.607567] Stack:
[13685.610774] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[13685.620056] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[13685.629345] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[13685.638639] Call Trace:
[13685.642364] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13685.651520] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13685.660678] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13685.669546] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13685.678209] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13685.685410] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13685.694758] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13685.701182] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13685.709467] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13685.716474] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13685.724736] Code: 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d
[13713.320184] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13713.329789] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13713.461453] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13713.474730] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13713.490960] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13713.500334] RIP: 0010:[<ffffffffa121ba26>] [<ffffffffa121ba26>] osd_scrub_exec+0x96/0x5b0 [osd_ldiskfs]
[13713.511973] RSP: 0018:ffff880703603d10 EFLAGS: 00000292
[13713.518905] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[13713.527872] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13713.536827] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13713.545771] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13713.554718] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[13713.563659] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13713.573683] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13713.581079] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13713.590040] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13713.599002] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13713.607943] Stack:
[13713.611150] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[13713.620441] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[13713.629747] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[13713.639037] Call Trace:
[13713.642754] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13713.652113] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13713.661278] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13713.670142] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13713.678813] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13713.686023] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13713.695365] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13713.701798] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13713.710077] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13713.717090] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13713.725371] Code: 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f 5d <c3> 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d 88
[13741.321167] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13741.330815] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13741.462640] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13741.475932] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13741.492197] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13741.501584] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[13741.513250] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[13741.520197] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[13741.529179] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13741.538153] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13741.547115] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13741.556068] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13741.565019] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13741.575057] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13741.582466] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13741.591444] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13741.600412] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13741.609361] Stack:
[13741.612582] ffffea00027e3c80 000000009f8f2001 0000000000000004 0000000000000001
[13741.621881] ffff880703603d57 ffff880703603d98 ffff880703603d78 ffff8800b2a36000
[13741.631189] ffff8803162f6000 ffffffffffffff10 ffffffffa1217a32 0000000000000010
[13741.640496] Call Trace:
[13741.644220] [<ffffffffa1217a32>] ? osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[13741.653013] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13741.662185] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13741.671345] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13741.680208] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13741.688888] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13741.696098] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13741.705461] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13741.711900] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13741.720178] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13741.727195] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13741.735474] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[13769.322143] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13769.331792] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13769.463570] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13769.476842] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13769.493081] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13769.502452] RIP: 0010:[<ffffffffa121ba1d>] [<ffffffffa121ba1d>] osd_scrub_exec+0x8d/0x5b0 [osd_ldiskfs]
[13769.514095] RSP: 0018:ffff880703603ce8 EFLAGS: 00000292
[13769.521031] RAX: 0000000000000000 RBX: ffff880703603c78 RCX: 0000000000000000
[13769.530004] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13769.538968] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13769.547921] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[13769.556870] R13: ffffffffa121ba87 R14: ffffffffffffff10 R15: ffff8803162f6000
[13769.565829] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13769.575856] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13769.583270] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13769.592233] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13769.601192] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13769.610131] Stack:
[13769.613358] ffff8800b2a3746c 0000000000000004 00000000ffffffff 0000000000000001
[13769.622659] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[13769.631961] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[13769.641248] Call Trace:
[13769.644974] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13769.654148] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13769.663303] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13769.672161] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13769.680830] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13769.688046] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13769.697392] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13769.703822] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13769.712111] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13769.719128] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13769.727397] Code: 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b <41> 5c 41 5d 41 5e 41 5f 5d c3 66 0f 1f 84 00 00 00 00 00 4c 8d
[13783.930657] INFO: rcu_sched self-detected stall on CPU { 24} (t=6360126 jiffies g=907112 c=907111 q=19635298)
[13783.943127] Task dump for CPU 24:
[13783.947863] OI_scrub R running task 0 5551 2 0x00000088
[13783.956793] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[13783.966125] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[13783.975421] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[13783.984707] Call Trace:
[13783.988403] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[13783.996378] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[13784.003348] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[13784.010870] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[13784.018568] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[13784.026818] [<ffffffff81099697>] update_process_times+0x47/0x80
[13784.034371] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[13784.042383] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[13784.049518] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[13784.057114] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[13784.064394] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[13784.071276] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[13784.079306] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[13784.087124] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[13784.094550] <EOI> [<ffffffffa121d00e>] ? osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[13784.104370] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13784.113476] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13784.122383] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13784.130998] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13784.139415] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13784.146367] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13784.155463] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13784.161639] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13784.169664] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13784.176425] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13809.323529] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13809.332951] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13809.462700] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13809.475886] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13809.491957] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13809.501261] RIP: 0010:[<ffffffffa121d00e>] [<ffffffffa121d00e>] osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[13809.513404] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[13809.520265] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[13809.529172] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[13809.538087] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[13809.546990] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13809.555881] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[13809.564775] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13809.574740] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13809.582084] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13809.590988] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13809.599893] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13809.608788] Stack:
[13809.611961] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13809.621200] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13809.630455] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13809.639708] Call Trace:
[13809.643382] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13809.652515] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13809.661365] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13809.670020] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13809.677211] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13809.686540] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13809.692946] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13809.701196] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13809.708185] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13809.716432] Code: 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 <66> 90 0f 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90
[13837.324505] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13837.334146] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13837.466086] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13837.479383] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13837.495670] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13837.505073] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[13837.516743] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[13837.523701] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[13837.532685] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13837.541660] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13837.550626] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13837.559578] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13837.568534] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13837.578561] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13837.585975] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13837.594943] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13837.603913] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13837.612870] Stack:
[13837.616084] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[13837.625376] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[13837.634659] ffff880703603d57 0000000000000004 0000000000000000 ffff880703603d78
[13837.643972] Call Trace:
[13837.647702] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13837.656865] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13837.666024] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13837.674891] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13837.683565] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13837.690768] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13837.700112] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13837.706544] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13837.714817] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13837.721822] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13837.730092] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[13865.325479] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13865.335106] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13865.466818] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13865.480106] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13865.496347] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13865.505727] RIP: 0010:[<ffffffffa121ba64>] [<ffffffffa121ba64>] osd_scrub_exec+0xd4/0x5b0 [osd_ldiskfs]
[13865.517374] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[13865.524313] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[13865.533286] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13865.542254] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13865.551200] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13865.560138] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13865.569075] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13865.579100] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13865.586501] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13865.595464] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13865.604415] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13865.613354] Stack:
[13865.616568] ffff880703603d08 0000000000000001 ffff880703603df0 000000009f8f2001
[13865.625855] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[13865.635140] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[13865.644420] Call Trace:
[13865.648130] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13865.657297] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13865.666452] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13865.675310] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13865.683963] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13865.691168] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13865.700517] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13865.706943] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13865.715215] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13865.722219] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13865.730481] Code: 4d 88 4c 89 ef e8 1d fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 <f6> 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00
[13893.326455] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13893.336147] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13893.467888] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13893.481171] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13893.497417] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13893.506807] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[13893.519035] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[13893.525983] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[13893.534961] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13893.543931] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13893.552888] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[13893.561834] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[13893.570789] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13893.580822] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13893.588226] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13893.597198] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13893.606151] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13893.615091] Stack:
[13893.618297] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13893.627598] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13893.636877] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13893.646157] Call Trace:
[13893.649870] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13893.659039] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13893.667917] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13893.676585] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13893.683790] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13893.693139] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13893.699579] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13893.707858] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13893.714873] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13893.723158] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[13921.327432] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13921.337135] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13921.468993] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13921.482310] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13921.498591] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13921.508007] RIP: 0010:[<ffffffffa1217b80>] [<ffffffffa1217b80>] osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[13921.519769] RSP: 0018:ffff880703603ce0 EFLAGS: 00000282
[13921.526729] RAX: 0000000000000004 RBX: ffff880703603c78 RCX: ffff880703603d98
[13921.535715] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[13921.544696] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[13921.553669] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[13921.562619] R13: ffffffffa121ba89 R14: ffffffffffffff10 R15: ffff8803162f6000
[13921.571571] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13921.581597] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13921.589007] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13921.597967] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13921.606929] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13921.615873] Stack:
[13921.619081] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[13921.628354] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[13921.637643] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[13921.646948] Call Trace:
[13921.650678] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[13921.659852] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13921.669006] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13921.677880] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13921.686557] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13921.693764] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13921.703108] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13921.709543] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13921.717821] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13921.724833] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13921.733101] Code: 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0
[13949.328405] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[13949.338087] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13949.469845] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13949.483133] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13949.499379] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13949.508763] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[13949.521002] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[13949.527945] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[13949.536924] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13949.545900] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[13949.554854] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[13949.563804] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[13949.572752] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13949.582786] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13949.590197] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13949.599162] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13949.608122] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13949.617064] Stack:
[13949.620275] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[13949.629576] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[13949.638878] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[13949.648190] Call Trace:
[13949.651907] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13949.661071] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13949.669940] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13949.678605] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13949.685807] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13949.695154] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13949.701589] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13949.709875] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13949.716884] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13949.725162] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[13963.939914] INFO: rcu_sched self-detected stall on CPU { 24} (t=6540129 jiffies g=907112 c=907111 q=21002172)
[13963.952393] Task dump for CPU 24:
[13963.957150] OI_scrub R running task 0 5551 2 0x00000088
[13963.966092] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[13963.975419] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[13963.984733] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[13963.994016] Call Trace:
[13963.997724] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[13964.005727] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[13964.012711] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[13964.020247] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[13964.027947] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[13964.036218] [<ffffffff81099697>] update_process_times+0x47/0x80
[13964.043781] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[13964.051815] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[13964.058950] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[13964.066548] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[13964.073837] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[13964.080717] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[13964.088752] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[13964.096580] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[13964.104002] <EOI> [<ffffffffa1162b53>] ? ldiskfs_itable_unused_count+0x23/0x30 [ldiskfs]
[13964.114031] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[13964.122940] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13964.131839] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13964.140449] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13964.148858] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13964.155808] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13964.164886] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13964.171051] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13964.179067] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13964.185823] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13989.329799] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[13989.339217] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[13989.468859] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[13989.482030] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[13989.498093] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[13989.507378] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[13989.518936] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[13989.525790] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[13989.534692] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[13989.543588] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[13989.552490] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[13989.561387] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[13989.570284] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[13989.580257] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[13989.587612] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[13989.596522] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[13989.605426] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[13989.614314] Stack:
[13989.617480] ffff880703603df0 000000009f8f2001 ffffea00027e3c80 000000009f8f2001
[13989.626731] ffff880703603d57 ffff880703603d57 ffff8800b2a37468 0000000000000000
[13989.636002] ffff880703603d78 ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[13989.645255] Call Trace:
[13989.648937] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[13989.658066] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[13989.667192] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[13989.676027] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[13989.684675] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[13989.691859] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[13989.701168] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[13989.707566] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13989.715814] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[13989.722788] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[13989.731026] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[14017.330775] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14017.340428] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14017.472161] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14017.485453] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14017.501705] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14017.511085] RIP: 0010:[<ffffffffa121ba26>] [<ffffffffa121ba26>] osd_scrub_exec+0x96/0x5b0 [osd_ldiskfs]
[14017.522726] RSP: 0018:ffff880703603d10 EFLAGS: 00000292
[14017.529652] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
[14017.538622] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14017.547580] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14017.556526] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[14017.565469] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[14017.574421] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14017.584439] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14017.591841] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14017.600796] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14017.609740] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14017.618671] Stack:
[14017.621869] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[14017.631129] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[14017.640417] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[14017.649696] Call Trace:
[14017.653413] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14017.662771] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14017.671933] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14017.680794] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14017.689457] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14017.696673] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14017.706021] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14017.712457] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14017.720742] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14017.727751] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14017.736019] Code: 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f 5d <c3> 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d 88
[14045.331751] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14045.341410] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14045.473342] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14045.486652] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14045.502904] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14045.512294] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[14045.523949] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[14045.530893] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[14045.539861] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14045.548825] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14045.557778] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14045.566730] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14045.575679] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14045.585717] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14045.593132] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14045.602113] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14045.611072] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14045.620028] Stack:
[14045.623264] 000000009f8f2001 0000000000000004 0000000000000001 ffffffffa1217a20
[14045.632574] ffff880703603d98 ffff880703603d57 ffff8800b2a36000 ffff8803162f6000
[14045.641873] ffffffffffffff10 ffffffffa1217a3b 0000000000000010 0000000000000287
[14045.651185] Call Trace:
[14045.654926] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14045.663807] [<ffffffffa1217a3b>] ? osd_scrub_next+0x1b/0x3c0 [osd_ldiskfs]
[14045.672606] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14045.681779] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14045.690948] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14045.699814] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14045.708497] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14045.715708] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14045.725070] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14045.731515] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14045.739814] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14045.746834] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14045.755123] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[14073.332724] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14073.342395] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14073.473932] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14073.487185] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14073.503388] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14073.512743] RIP: 0010:[<ffffffffa121b990>] [<ffffffffa121b990>] osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14073.524765] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[14073.531691] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[14073.540656] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14073.549615] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14073.558577] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[14073.567543] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[14073.576500] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14073.586541] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14073.593947] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14073.602908] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14073.611862] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14073.620813] Stack:
[14073.624040] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[14073.633334] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[14073.642635] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[14073.651932] Call Trace:
[14073.655654] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14073.665012] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14073.674169] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14073.683035] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14073.691712] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14073.698927] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14073.708284] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14073.714723] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14073.723016] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14073.730039] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14073.738317] Code: b5 07 00 00 c7 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45
[14101.333701] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14101.343380] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14101.475521] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14101.488836] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14101.505119] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14101.514518] RIP: 0010:[<ffffffffa121ba64>] [<ffffffffa121ba64>] osd_scrub_exec+0xd4/0x5b0 [osd_ldiskfs]
[14101.526185] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[14101.533140] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[14101.542133] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14101.551106] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14101.560061] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14101.569024] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14101.577988] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14101.588013] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14101.595424] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14101.604409] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14101.613372] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14101.622327] Stack:
[14101.625557] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 0000000000000001
[14101.634860] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[14101.644154] ffffffffa121b990 0000000000000000 ffff880703603d78 ffff8800b2a36000
[14101.653456] Call Trace:
[14101.657176] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14101.666343] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14101.675504] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14101.684656] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14101.693515] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14101.702181] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14101.709388] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14101.718727] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14101.725149] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14101.733439] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14101.740446] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14101.748707] Code: 4d 88 4c 89 ef e8 1d fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 <f6> 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00
[14129.334675] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14129.344350] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14129.475910] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14129.489170] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14129.505369] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14129.514732] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[14129.526365] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[14129.533294] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[14129.542259] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14129.551216] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14129.560163] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14129.569096] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14129.578052] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14129.588080] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14129.595481] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14129.604434] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14129.613386] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14129.622320] Stack:
[14129.625527] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[14129.634820] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[14129.644107] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[14129.653381] Call Trace:
[14129.657096] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14129.666253] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14129.675400] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14129.684257] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14129.692923] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14129.700126] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14129.709469] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14129.715887] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14129.724183] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14129.731194] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14129.739459] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[14143.949185] INFO: rcu_sched self-detected stall on CPU { 24} (t=6720132 jiffies g=907112 c=907111 q=21876797)
[14143.961685] Task dump for CPU 24:
[14143.966412] OI_scrub R running task 0 5551 2 0x00000088
[14143.975322] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[14143.984639] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[14143.993933] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[14144.003202] Call Trace:
[14144.006869] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[14144.014844] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[14144.021797] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[14144.029310] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[14144.036996] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[14144.045242] [<ffffffff81099697>] update_process_times+0x47/0x80
[14144.052792] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[14144.060808] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[14144.067933] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[14144.075529] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[14144.082813] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[14144.089689] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[14144.097717] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[14144.105541] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[14144.112964] <EOI> [<ffffffffa1217b69>] ? osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[14144.122291] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14144.130900] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14144.139801] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14144.148705] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14144.157308] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14144.165702] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14144.172637] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14144.181722] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14144.187892] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14144.195904] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14144.202644] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14169.336075] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14169.345473] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14169.475607] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14169.488783] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14169.504823] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14169.514103] RIP: 0010:[<ffffffffa121d010>] [<ffffffffa121d010>] osd_inode_iteration+0x2c0/0xcc0 [osd_ldiskfs]
[14169.526251] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[14169.533114] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[14169.542016] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[14169.550919] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[14169.559819] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[14169.568710] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[14169.577596] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14169.587558] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14169.594899] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14169.603813] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14169.612717] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14169.621612] Stack:
[14169.624785] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14169.634025] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14169.643279] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14169.652517] Call Trace:
[14169.656205] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14169.665351] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14169.674189] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14169.682842] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14169.690022] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14169.699351] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14169.705746] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14169.714007] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14169.720989] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14169.729238] Code: 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 <0f> 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90 48 85
[14197.337049] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14197.346643] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14197.478574] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14197.491882] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14197.508175] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14197.517579] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[14197.529244] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[14197.536205] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[14197.545192] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14197.554178] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14197.563147] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14197.572099] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14197.581048] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14197.591076] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14197.598487] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14197.607460] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14197.616431] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14197.625393] Stack:
[14197.628615] ffff880703603d08 ffff880703603d57 ffff880703603df0 000000009f8f2001
[14197.637902] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[14197.647198] 0000000000000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[14197.656493] Call Trace:
[14197.660216] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14197.669390] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14197.678561] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14197.687433] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14197.696120] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14197.703345] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14197.712720] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14197.719156] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14197.727453] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14197.734478] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14197.742760] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[14225.338019] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14225.347633] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14225.479386] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14225.492674] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14225.508936] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14225.518320] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[14225.530553] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[14225.537494] RAX: ffff88042bd78800 RBX: ffff880703603d57 RCX: ffff88042bd78800
[14225.546482] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[14225.555448] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14225.564408] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[14225.573359] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[14225.582310] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14225.592340] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14225.599748] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14225.608719] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14225.617680] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14225.626624] Stack:
[14225.629840] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14225.639132] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14225.648434] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14225.657739] Call Trace:
[14225.661461] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14225.670623] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14225.679493] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14225.688167] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14225.695373] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14225.704734] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14225.711172] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14225.719455] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14225.726467] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14225.734745] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[14253.339001] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14253.348632] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14253.481082] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14253.494396] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14253.510694] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14253.520104] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[14253.532371] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[14253.539334] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[14253.548330] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14253.557320] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14253.566291] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[14253.575249] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[14253.584201] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14253.594233] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14253.601641] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14253.610618] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14253.619585] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14253.628537] Stack:
[14253.631754] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14253.641058] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14253.650337] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14253.659634] Call Trace:
[14253.663351] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14253.672521] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14253.681398] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14253.690078] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14253.697296] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14253.706657] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14253.713091] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14253.721392] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14253.728410] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14253.736684] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[14281.339977] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14281.349598] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14281.481457] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14281.494765] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14281.511067] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14281.520472] RIP: 0010:[<ffffffffa121cfe9>] [<ffffffffa121cfe9>] osd_inode_iteration+0x299/0xcc0 [osd_ldiskfs]
[14281.532732] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[14281.539697] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff880703603d98
[14281.548690] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff88042bd79800
[14281.557673] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[14281.566640] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[14281.575596] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[14281.584549] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14281.594576] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14281.601994] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14281.610967] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14281.619937] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14281.628892] Stack:
[14281.632112] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14281.641409] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14281.650698] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14281.659996] Call Trace:
[14281.663724] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14281.672897] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14281.681779] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14281.690461] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14281.697683] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14281.707041] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14281.713471] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14281.721756] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14281.728783] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14281.737070] Code: 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89
[14309.340950] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14309.350596] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14309.482796] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14309.496111] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14309.512426] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14309.521836] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[14309.533519] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[14309.540479] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[14309.549474] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14309.558454] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14309.567424] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14309.576384] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14309.585340] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14309.595372] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14309.602779] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14309.611749] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14309.620712] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14309.629659] Stack:
[14309.632878] ffff880703603d08 000000009f8f2001 ffffea00027e3c80 000000009f8f2001
[14309.642166] ffff880703603d57 0000000000000001 ffffffffa1217a20 ffff880703603d98
[14309.651456] ffff880703603d78 ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10
[14309.660754] Call Trace:
[14309.664485] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14309.673361] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14309.682531] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14309.691696] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14309.700562] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14309.709234] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14309.716441] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14309.725800] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14309.732230] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14309.740511] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14309.747519] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14309.755788] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[14323.958461] INFO: rcu_sched self-detected stall on CPU { 24} (t=6900135 jiffies g=907112 c=907111 q=22047866)
[14323.970884] Task dump for CPU 24:
[14323.975601] OI_scrub R running task 0 5551 2 0x00000088
[14323.984513] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[14323.993812] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[14324.003082] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[14324.012325] Call Trace:
[14324.015997] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[14324.023953] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[14324.030891] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[14324.038389] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[14324.046059] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[14324.054289] [<ffffffff81099697>] update_process_times+0x47/0x80
[14324.061810] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[14324.069805] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[14324.076905] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[14324.084478] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[14324.091747] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[14324.098609] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[14324.106614] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[14324.114437] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[14324.121863] <EOI> [<ffffffffa121ba25>] ? osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[14324.131101] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14324.140008] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14324.148913] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14324.157513] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14324.165910] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14324.172850] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14324.181939] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14324.188111] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14324.196124] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14324.202867] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14349.342345] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14349.351714] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14349.481628] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14349.494800] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14349.510864] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14349.520161] RIP: 0010:[<ffffffffa1217d70>] [<ffffffffa1217d70>] osd_scrub_next+0x350/0x3c0 [osd_ldiskfs]
[14349.531818] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[14349.538676] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[14349.547570] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14349.556467] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14349.565369] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14349.574268] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14349.583163] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14349.593124] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14349.600453] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14349.609361] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14349.618267] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14349.627168] Stack:
[14349.630338] 000000009f8f2001 ffff880703603df0 000000009f8f2001 ffff880703603df0
[14349.639572] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[14349.648808] ffff880703603d57 ffff8800b2a37468 0000000000000000 ffff880703603d78
[14349.658057] Call Trace:
[14349.661738] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14349.670864] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14349.679991] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14349.688819] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14349.697452] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14349.704624] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14349.713934] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14349.720331] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14349.728566] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14349.735539] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14349.743780] Code: 97 ff 48 8b 55 90 48 8b 4d 88 44 8b 85 74 ff ff ff e9 1d ff ff ff 48 89 d7 44 89 45 90 e8 f9 67 47 e0 44 8b 45 90 0f 1f 44 00 00 <45> 84 c0 b8 04 00 00 00 0f 85 eb fd ff ff 49 8d b7 6c 14 00 00
[14377.343322] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14377.352942] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14377.484694] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14377.497974] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14377.514216] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14377.523601] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[14377.535246] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[14377.542178] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[14377.551148] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14377.560105] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14377.569055] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[14377.578006] R13: 0000000000000296 R14: 0000000000000010 R15: ffffffffa121b9ac
[14377.586948] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14377.596967] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14377.604364] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14377.613317] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14377.622260] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14377.631191] Stack:
[14377.634389] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[14377.643674] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[14377.652968] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[14377.662266] Call Trace:
[14377.665977] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14377.675144] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14377.684304] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14377.693168] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14377.701836] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14377.709044] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14377.718390] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14377.724826] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14377.733104] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14377.740114] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14377.748381] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[14405.344290] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14405.353949] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14405.485908] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14405.499194] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14405.515458] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14405.524852] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[14405.537094] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[14405.544050] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[14405.553032] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[14405.561997] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14405.570952] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[14405.579903] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[14405.588859] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14405.598893] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14405.606294] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14405.615265] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14405.624221] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14405.633165] Stack:
[14405.636384] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14405.645679] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14405.654980] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14405.664265] Call Trace:
[14405.667991] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14405.677162] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14405.686039] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14405.694710] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14405.701913] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14405.711262] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14405.717693] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14405.725977] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14405.732988] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14405.741265] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[14433.345272] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14433.354969] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14433.486867] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14433.500179] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14433.516466] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14433.525873] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[14433.538132] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[14433.545101] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[14433.554096] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14433.563076] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14433.572040] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[14433.580990] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffff880703603d08
[14433.589943] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14433.599967] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14433.607373] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14433.616346] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14433.625313] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14433.634270] Stack:
[14433.637501] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14433.646783] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14433.656078] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14433.665376] Call Trace:
[14433.669103] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14433.678262] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14433.687139] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14433.695829] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14433.703046] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14433.712398] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14433.718835] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14433.727117] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14433.734132] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14433.742407] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[14461.346255] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14461.355930] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14461.487955] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14461.501282] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14461.517585] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14461.527000] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[14461.538669] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[14461.545641] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[14461.554637] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14461.563619] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14461.572588] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[14461.581543] R13: ffff880703603d08 R14: 0000000000000010 R15: ffffffffa121ba87
[14461.590487] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14461.600516] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14461.607936] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14461.616915] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14461.625888] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14461.634852] Stack:
[14461.638067] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[14461.647341] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[14461.656636] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[14461.665943] Call Trace:
[14461.669682] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14461.678855] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14461.688030] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14461.696917] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14461.705592] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14461.712811] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14461.722177] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14461.728620] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14461.736902] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14461.743919] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14461.752203] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[14489.347216] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14489.356851] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14489.488705] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14489.501998] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14489.518268] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14489.527665] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[14489.539335] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[14489.546276] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[14489.555263] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14489.564232] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14489.573197] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14489.582157] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14489.591119] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14489.601152] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14489.608570] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14489.617539] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14489.626498] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14489.635451] Stack:
[14489.638666] ffffea00027e3c80 000000009f8f2001 0000000000000004 0000000000000001
[14489.647960] ffff880703603d57 ffff880703603d98 ffff880703603d78 ffff8800b2a36000
[14489.657279] ffff8803162f6000 ffffffffffffff10 ffffffffa1217a32 0000000000000010
[14489.666589] Call Trace:
[14489.670324] [<ffffffffa1217a32>] ? osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[14489.679110] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14489.688277] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14489.697437] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14489.706296] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14489.714969] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14489.722177] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14489.731534] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14489.737971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14489.746256] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14489.753263] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14489.761533] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[14503.967727] INFO: rcu_sched self-detected stall on CPU { 24} (t=7080138 jiffies g=907112 c=907111 q=22723963)
[14503.980179] Task dump for CPU 24:
[14503.984897] OI_scrub R running task 0 5551 2 0x00000088
[14503.993812] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[14504.003106] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[14504.012381] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[14504.021660] Call Trace:
[14504.025340] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[14504.033304] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[14504.040244] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[14504.047742] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[14504.055413] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[14504.063658] [<ffffffff81099697>] update_process_times+0x47/0x80
[14504.071188] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[14504.079193] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[14504.086301] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[14504.093867] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[14504.101134] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[14504.108012] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[14504.116028] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[14504.123860] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[14504.131287] <EOI> [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14504.140914] [<ffffffffa121b9b3>] ? osd_scrub_exec+0x23/0x5b0 [osd_ldiskfs]
[14504.149433] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14504.158343] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14504.167239] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14504.175845] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14504.184255] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14504.191211] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14504.200312] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14504.206491] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14504.214504] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14504.221257] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14529.348616] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14529.358036] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14529.488094] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14529.501273] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14529.517317] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14529.526612] RIP: 0010:[<ffffffffa1217b69>] [<ffffffffa1217b69>] osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[14529.538276] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[14529.545146] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[14529.554047] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14529.562949] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14529.571849] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14529.580746] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14529.589644] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14529.599611] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14529.606960] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14529.615874] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14529.624785] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14529.633699] Stack:
[14529.636879] 000000009f8f2001 ffff880703603d08 ffff880703603d57 ffff880703603df0
[14529.646129] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[14529.655382] ffff880703603d57 ffff8800b2a37468 0000000000000000 ffff880703603d78
[14529.664641] Call Trace:
[14529.668345] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14529.677485] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14529.686621] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14529.695468] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14529.704112] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14529.711291] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14529.720611] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14529.727014] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14529.735277] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14529.742260] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14529.750507] Code: 45 90 74 49 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 <48> 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83
[14557.349584] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14557.359196] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14557.491054] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14557.504344] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14557.520614] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14557.530008] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[14557.541666] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[14557.548615] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[14557.557596] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14557.566566] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14557.575521] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14557.584476] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14557.593424] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14557.603458] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14557.610868] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14557.619839] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14557.628790] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14557.637737] Stack:
[14557.640940] ffff880703603d08 000000009f8f2001 ffffea00027e3c80 000000009f8f2001
[14557.650243] ffff880703603d57 ffff880703603d57 0000000000000000 0000000000000000
[14557.659546] ffff880703603d78 ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10
[14557.668856] Call Trace:
[14557.672572] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14557.681743] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14557.690907] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14557.699773] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14557.708434] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14557.715645] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14557.725004] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14557.731436] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14557.739729] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14557.746741] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14557.755012] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[14585.350568] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14585.360283] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14585.492107] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14585.505395] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14585.521652] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14585.531039] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[14585.542694] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[14585.549635] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[14585.558612] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14585.567581] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14585.576534] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14585.585487] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14585.594438] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14585.604462] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14585.611864] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14585.620832] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14585.629788] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14585.638725] Stack:
[14585.641937] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[14585.651241] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[14585.660525] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[14585.669828] Call Trace:
[14585.673543] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14585.682713] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14585.691881] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14585.700744] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14585.709415] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14585.716624] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14585.725996] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14585.732434] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14585.740725] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14585.747732] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14585.755999] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[14613.351543] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14613.361201] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14613.493418] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14613.506717] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14613.522993] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14613.532384] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[14613.544634] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[14613.551590] RAX: 0000000020000000 RBX: ffff880703603d98 RCX: 0000000000000000
[14613.560571] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14613.569537] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14613.578501] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[14613.587449] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[14613.596405] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14613.606437] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14613.613845] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14613.622813] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14613.631770] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14613.640716] Stack:
[14613.643928] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14613.653213] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14613.662488] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14613.671801] Call Trace:
[14613.675532] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14613.684689] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14613.693557] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14613.702232] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14613.709442] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14613.718779] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14613.725213] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14613.733506] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14613.740520] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14613.748802] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[14641.352518] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14641.362160] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14641.494132] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14641.507438] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14641.523721] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14641.533125] RIP: 0010:[<ffffffffa1217b6d>] [<ffffffffa1217b6d>] osd_scrub_next+0x14d/0x3c0 [osd_ldiskfs]
[14641.544883] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[14641.551843] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[14641.560821] RDX: ffff880703603d78 RSI: 00000000fb6db880 RDI: ffff8803162f6000
[14641.569800] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14641.578768] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14641.587722] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14641.596664] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14641.606691] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14641.614100] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14641.623070] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14641.632036] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14641.640990] Stack:
[14641.644208] 000000009f8f2001 ffff880703603d08 000000009f8f2001 ffff880703603df0
[14641.653493] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[14641.662783] 0000000000000001 0000000000000004 ffff880703603d98 ffff880703603d78
[14641.672077] Call Trace:
[14641.675801] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14641.684968] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14641.694120] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14641.702989] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14641.711665] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14641.718870] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14641.728213] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14641.734633] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14641.742902] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14641.749917] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14641.758183] Code: 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 <65> 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41
[14669.353493] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14669.363149] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14669.495549] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14669.508845] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14669.525106] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14669.534487] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[14669.546151] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[14669.553085] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[14669.562063] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14669.571028] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14669.579981] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[14669.588940] R13: 0000000000000246 R14: 0000000000000010 R15: ffffffffa121ba87
[14669.597896] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14669.607913] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14669.615319] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14669.624284] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14669.633241] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14669.642180] Stack:
[14669.645392] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[14669.654672] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[14669.663962] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[14669.673254] Call Trace:
[14669.676980] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14669.686154] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14669.695320] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14669.704181] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14669.712855] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14669.720071] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14669.729433] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14669.735870] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14669.744163] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14669.751180] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14669.759460] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[14672.578441] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[14672.593356] Lustre: Skipped 1 previous similar message
[14672.601615] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[14672.616475] Lustre: Skipped 1 previous similar message
[14683.976997] INFO: rcu_sched self-detected stall on CPU { 24} (t=7260141 jiffies g=907112 c=907111 q=22990120)
[14683.989488] Task dump for CPU 24:
[14683.994222] OI_scrub R running task 0 5551 2 0x00000088
[14684.003151] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[14684.012464] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[14684.021745] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[14684.031025] Call Trace:
[14684.034697] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[14684.042657] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[14684.049594] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[14684.057092] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[14684.064770] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[14684.073018] [<ffffffff81099697>] update_process_times+0x47/0x80
[14684.080556] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[14684.088565] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[14684.095693] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[14684.103284] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[14684.110587] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[14684.117501] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[14684.125572] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[14684.133446] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[14684.140937] <EOI> [<ffffffffa1217a65>] ? osd_scrub_next+0x45/0x3c0 [osd_ldiskfs]
[14684.150233] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14684.158890] [<ffffffffa1217a3b>] ? osd_scrub_next+0x1b/0x3c0 [osd_ldiskfs]
[14684.167448] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14684.176398] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14684.185345] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14684.194000] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14684.202463] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14684.209466] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14684.218597] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14684.224813] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14684.232861] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14684.239652] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14709.354887] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14709.364386] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14709.494532] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14709.507688] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14709.523685] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14709.532949] RIP: 0010:[<ffffffffa1217b6d>] [<ffffffffa1217b6d>] osd_scrub_next+0x14d/0x3c0 [osd_ldiskfs]
[14709.544594] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[14709.551443] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[14709.560337] RDX: ffff880703603d78 RSI: 00000000fb6db880 RDI: ffff8803162f6000
[14709.569233] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14709.578129] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14709.587026] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14709.595912] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14709.605875] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14709.613223] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14709.622133] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14709.631038] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14709.639935] Stack:
[14709.643122] 000000009f8f2001 ffffea00027e3c80 000000009f8f2001 ffff880703603df0
[14709.652365] 0000000000000001 ffff880703603d57 000000009f8f2001 0000000000000004
[14709.661598] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[14709.670842] Call Trace:
[14709.674525] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14709.683666] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14709.692790] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14709.701627] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14709.710254] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14709.717425] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14709.726734] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14709.733129] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14709.741382] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14709.748361] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14709.756600] Code: 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 <65> 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41
[14737.355856] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14737.365496] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14737.497481] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14737.510766] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14737.527016] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14737.536407] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[14737.548063] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[14737.555000] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[14737.563975] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14737.572949] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14737.581913] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14737.590867] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14737.599823] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14737.609856] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14737.617259] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14737.626221] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14737.635172] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14737.644111] Stack:
[14737.647322] ffff880703603d08 0000000000000001 ffff880703603df0 000000009f8f2001
[14737.656626] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[14737.665924] 0000000000000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[14737.675211] Call Trace:
[14737.678939] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14737.688103] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14737.697263] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14737.706129] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14737.714789] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14737.721995] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14737.731347] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14737.737787] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14737.746063] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14737.753070] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14737.761335] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[14765.356831] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14765.366532] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14765.498444] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14765.511729] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14765.527975] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14765.537363] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[14765.549598] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[14765.556544] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[14765.565528] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14765.574498] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14765.583451] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[14765.592405] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[14765.601351] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14765.611373] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14765.618764] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14765.627730] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14765.636694] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14765.645630] Stack:
[14765.648845] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14765.658138] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14765.667434] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14765.676725] Call Trace:
[14765.680436] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14765.689593] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14765.698459] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14765.707124] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14765.714327] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14765.723672] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14765.730114] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14765.738411] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14765.745420] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14765.753696] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[14793.357808] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14793.367476] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14793.499468] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14793.512773] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14793.529071] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14793.538490] RIP: 0010:[<ffffffffa121b990>] [<ffffffffa121b990>] osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14793.550551] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[14793.557515] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[14793.566499] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14793.575487] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14793.584450] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[14793.593399] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffffea00027e3c80
[14793.602348] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14793.612382] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14793.619796] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14793.628772] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14793.637739] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14793.646706] Stack:
[14793.649928] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[14793.659222] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[14793.668518] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[14793.677820] Call Trace:
[14793.681558] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14793.690926] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14793.700101] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14793.708990] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14793.717668] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14793.724886] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14793.734243] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14793.740684] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14793.748971] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14793.755990] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14793.764266] Code: b5 07 00 00 c7 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45
[14821.358790] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14821.368434] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14821.500464] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14821.513763] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14821.530029] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14821.539429] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[14821.551690] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[14821.558648] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[14821.567643] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[14821.576624] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14821.585580] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[14821.594544] R13: ffff880703603df0 R14: 000000009f8f2001 R15: 0000000000000018
[14821.603494] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14821.613529] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14821.620947] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14821.629928] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14821.638896] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14821.647846] Stack:
[14821.651061] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[14821.660367] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[14821.669665] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[14821.678965] Call Trace:
[14821.682702] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14821.691877] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14821.700749] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14821.709441] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14821.716655] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14821.726005] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14821.732443] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14821.740729] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14821.747747] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14821.756027] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[14849.359756] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14849.369381] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14849.501612] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14849.514918] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14849.531219] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14849.540628] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[14849.552304] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[14849.559277] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[14849.568269] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14849.577251] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14849.586218] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14849.595175] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14849.604129] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14849.614162] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14849.621574] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14849.630544] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14849.639512] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14849.648471] Stack:
[14849.651689] ffff880703603d08 0000000000000001 ffff880703603df0 000000009f8f2001
[14849.660985] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[14849.670284] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[14849.679594] Call Trace:
[14849.683338] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14849.692518] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14849.701679] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14849.710556] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14849.719243] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14849.726457] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14849.735823] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14849.742250] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14849.750545] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14849.757559] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14849.765838] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[14863.986281] INFO: rcu_sched self-detected stall on CPU { 24} (t=7440144 jiffies g=907112 c=907111 q=23642993)
[14863.998731] Task dump for CPU 24:
[14864.003463] OI_scrub R running task 0 5551 2 0x00000088
[14864.012388] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[14864.021719] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[14864.031020] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[14864.040330] Call Trace:
[14864.044012] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[14864.052002] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[14864.058957] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[14864.066479] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[14864.074168] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[14864.082427] [<ffffffff81099697>] update_process_times+0x47/0x80
[14864.089976] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[14864.097995] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[14864.105117] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[14864.112702] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[14864.119990] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[14864.126872] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[14864.134912] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[14864.142732] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[14864.150163] <EOI> [<ffffffffa121ba25>] ? osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[14864.159391] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14864.168296] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14864.177206] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14864.185820] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14864.194210] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14864.201152] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14864.210234] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14864.216406] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14864.224439] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14864.231192] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14889.361151] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14889.370601] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14889.500301] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14889.513469] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14889.529526] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14889.538814] RIP: 0010:[<ffffffffa121b990>] [<ffffffffa121b990>] osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14889.550761] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[14889.557622] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[14889.566521] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14889.575414] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[14889.584308] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[14889.593205] R13: ffff880703603df0 R14: 000000009f8f2001 R15: 0000000000000018
[14889.602102] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14889.612065] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14889.619399] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14889.628301] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14889.637203] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14889.646092] Stack:
[14889.649259] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[14889.658497] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[14889.667741] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[14889.676993] Call Trace:
[14889.680679] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14889.690009] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14889.699137] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14889.707986] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14889.716637] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14889.723825] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14889.733144] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14889.739552] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14889.747792] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14889.754781] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14889.763024] Code: b5 07 00 00 c7 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45
[14917.362133] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14917.371761] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14917.503902] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14917.517193] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14917.533471] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14917.542865] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[14917.554533] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[14917.561476] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[14917.570462] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14917.579429] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[14917.588391] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14917.597344] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14917.606316] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14917.616345] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14917.623761] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14917.632740] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14917.641710] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14917.650673] Stack:
[14917.653895] 000000009f8f2001 ffffea00027e3c80 000000009f8f2001 ffff880703603df0
[14917.663212] ffff880703603d57 ffff880703603d57 000000009f8f2001 0000000000000004
[14917.672523] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[14917.681819] Call Trace:
[14917.685554] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[14917.694718] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14917.703886] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14917.712757] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14917.721424] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14917.728627] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14917.737998] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14917.744424] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14917.752724] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14917.759741] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14917.768013] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[14945.363101] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[14945.372754] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14945.505026] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14945.518323] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14945.534588] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14945.543968] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[14945.555625] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[14945.562559] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[14945.571532] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14945.580494] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14945.589442] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[14945.598395] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[14945.607356] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14945.617388] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14945.624794] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14945.633754] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14945.642717] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14945.651669] Stack:
[14945.654887] ffff880703603d08 000000009f8f2001 ffff880703603df0 000000009f8f2001
[14945.664172] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[14945.673471] 0000000000000004 0000000000000000 ffff880703603d78 ffff8800b2a36000
[14945.682759] Call Trace:
[14945.686483] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14945.695640] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14945.704805] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14945.713664] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14945.722331] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14945.729534] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14945.738886] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14945.745312] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14945.753584] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14945.760594] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14945.768866] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[14973.364076] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[14973.373687] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[14973.505555] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[14973.518837] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[14973.535069] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[14973.544450] RIP: 0010:[<ffffffffa121b9a4>] [<ffffffffa121b9a4>] osd_scrub_exec+0x14/0x5b0 [osd_ldiskfs]
[14973.556110] RSP: 0018:ffff880703603ce8 EFLAGS: 00000293
[14973.563053] RAX: ffffffffa121b990 RBX: ffff880703603c78 RCX: 0000000000000000
[14973.572032] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[14973.580999] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[14973.589954] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[14973.598887] R13: ffffffffa121b9ac R14: ffffffffffffff10 R15: ffff8803162f6000
[14973.607836] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[14973.617865] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[14973.625271] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[14973.634236] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[14973.643189] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[14973.652131] Stack:
[14973.655338] ffff8800b2a3746c 0000000000000004 00000000ffffffff 0000000000000001
[14973.664615] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[14973.673922] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[14973.683215] Call Trace:
[14973.686941] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[14973.696111] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[14973.705269] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[14973.714140] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[14973.722805] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[14973.730019] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[14973.739386] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[14973.745821] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14973.754109] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[14973.761130] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[14973.769405] Code: 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 <45> 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48
[15001.365056] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15001.374698] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15001.506578] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15001.519876] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15001.536144] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15001.545527] RIP: 0010:[<ffffffffa121d1fc>] [<ffffffffa121d1fc>] osd_inode_iteration+0x4ac/0xcc0 [osd_ldiskfs]
[15001.557770] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[15001.564719] RAX: 0000000020000000 RBX: 0000000000000000 RCX: 0000000000000000
[15001.573703] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15001.582677] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15001.591639] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[15001.600597] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[15001.609555] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15001.619594] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15001.627016] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15001.635989] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15001.644957] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15001.653908] Stack:
[15001.657124] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15001.666433] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15001.675721] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15001.685015] Call Trace:
[15001.688746] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15001.697917] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15001.706792] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15001.715460] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15001.722671] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15001.732033] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15001.738471] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15001.746755] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15001.753773] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15001.762051] Code: 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 <48> 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2
[15029.366035] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15029.375672] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15029.508014] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15029.521315] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15029.537628] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15029.547026] RIP: 0010:[<ffffffffa121ba6b>] [<ffffffffa121ba6b>] osd_scrub_exec+0xdb/0x5b0 [osd_ldiskfs]
[15029.558713] RSP: 0018:ffff880703603c78 EFLAGS: 00000202
[15029.565672] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[15029.574661] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15029.583641] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15029.592602] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15029.601555] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15029.610498] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15029.620526] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15029.627927] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15029.636900] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15029.645862] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15029.654813] Stack:
[15029.658028] ffff880703603d08 0000000000000001 ffffea00027e3c80 000000009f8f2001
[15029.667319] ffff880703603d57 ffff880703603d57 0000000000000000 ffff88042bd78800
[15029.676625] 00000000000072ce ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[15029.685913] Call Trace:
[15029.689642] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15029.698820] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15029.707982] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15029.716857] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15029.725531] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15029.732734] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15029.742098] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15029.748536] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15029.756838] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15029.763849] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15029.772127] Code: fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 <74> 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00
[15043.995544] INFO: rcu_sched self-detected stall on CPU { 24} (t=7620147 jiffies g=907112 c=907111 q=23979408)
[15044.008013] Task dump for CPU 24:
[15044.012739] OI_scrub R running task 0 5551 2 0x00000088
[15044.021675] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[15044.031012] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[15044.040305] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[15044.049583] Call Trace:
[15044.053263] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[15044.061240] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[15044.068208] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[15044.075729] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[15044.083422] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[15044.091686] [<ffffffff81099697>] update_process_times+0x47/0x80
[15044.099242] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[15044.107254] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[15044.114375] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[15044.121960] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[15044.129254] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[15044.136141] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[15044.144178] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[15044.152021] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[15044.159456] <EOI> [<ffffffffa121cfe9>] ? osd_inode_iteration+0x299/0xcc0 [osd_ldiskfs]
[15044.169288] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[15044.178395] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15044.187301] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15044.195928] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15044.204348] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15044.211302] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15044.220420] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15044.226598] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15044.234630] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15044.241398] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15046.650465] Lustre: soaked-MDT0000: Client 40e8ebdf-e84f-0dfe-7f82-ddc39c67a41e (at 192.168.1.132@o2ib100) reconnecting
[15046.663512] Lustre: soaked-MDT0000: Connection restored to 7eb6195d-0392-6b9e-dc88-22a505b06cf8 (at 192.168.1.132@o2ib100)
[15069.367430] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15069.376909] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15069.506938] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15069.520136] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15069.536240] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15069.545545] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[15069.557721] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[15069.564621] RAX: ffff88042bd78800 RBX: ffff880703603d98 RCX: ffff88042bd78800
[15069.573567] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[15069.582516] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15069.591451] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[15069.600379] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[15069.609304] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15069.619311] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15069.626702] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15069.635652] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15069.644599] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15069.653533] Stack:
[15069.656739] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15069.666041] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15069.675330] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15069.684622] Call Trace:
[15069.688353] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15069.697524] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15069.706401] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15069.715084] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15069.722302] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15069.731665] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15069.738104] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15069.746392] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15069.753404] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15069.761685] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[15097.368403] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15097.378051] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15097.510234] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15097.523546] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15097.539859] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15097.549268] RIP: 0010:[<ffffffffa1217a29>] [<ffffffffa1217a29>] osd_scrub_next+0x9/0x3c0 [osd_ldiskfs]
[15097.560857] RSP: 0018:ffff880703603d08 EFLAGS: 00000287
[15097.567821] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[15097.576810] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15097.585793] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[15097.594765] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15097.603719] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[15097.612669] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15097.622686] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15097.630088] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15097.639051] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15097.648017] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15097.656978] Stack:
[15097.660204] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[15097.669493] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[15097.678779] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[15097.688058] Call Trace:
[15097.691783] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15097.700955] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15097.710110] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15097.718981] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15097.727659] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15097.734866] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15097.744234] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15097.750662] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15097.758945] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15097.765953] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15097.774220] Code: 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 <41> 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb
[15125.369373] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15125.379046] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15125.511017] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15125.524311] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15125.540544] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15125.549920] RIP: 0010:[<ffffffffa1162b36>] [<ffffffffa1162b36>] ldiskfs_itable_unused_count+0x6/0x30 [ldiskfs]
[15125.562246] RSP: 0018:ffff880703603d08 EFLAGS: 00000286
[15125.569179] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[15125.578157] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[15125.587123] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15125.596072] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15125.605016] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000202
[15125.613964] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15125.623994] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15125.631398] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15125.640367] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15125.649319] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15125.658257] Stack:
[15125.661464] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[15125.670749] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[15125.680044] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[15125.689342] Call Trace:
[15125.693077] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[15125.702442] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15125.711611] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15125.720471] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15125.729148] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15125.736356] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15125.745701] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15125.752136] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15125.760418] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15125.767430] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15125.775704] Code: 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 <48> 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76
[15153.370348] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15153.380017] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15153.511951] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15153.525238] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15153.541496] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15153.550891] RIP: 0010:[<ffffffffa1217a35>] [<ffffffffa1217a35>] osd_scrub_next+0x15/0x3c0 [osd_ldiskfs]
[15153.562549] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[15153.569494] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[15153.578465] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15153.587434] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[15153.596390] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[15153.605343] R13: 0000000000000246 R14: 0000000000000010 R15: ffffffffa121ba12
[15153.614296] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15153.624326] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15153.631734] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15153.640705] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15153.649674] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15153.658631] Stack:
[15153.661856] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[15153.671147] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[15153.680447] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[15153.689732] Call Trace:
[15153.693472] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15153.702655] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15153.711828] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15153.720703] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15153.729385] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15153.736592] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15153.745951] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15153.752394] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15153.760678] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15153.767699] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15153.775982] Code: ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd <41> 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00
[15181.371329] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15181.380979] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15181.513303] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15181.526588] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15181.542859] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15181.552249] RIP: 0010:[<ffffffffa121ba1d>] [<ffffffffa121ba1d>] osd_scrub_exec+0x8d/0x5b0 [osd_ldiskfs]
[15181.563916] RSP: 0018:ffff880703603ce8 EFLAGS: 00000292
[15181.570861] RAX: 0000000000000000 RBX: ffff880703603c78 RCX: 0000000000000000
[15181.579852] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15181.588822] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15181.597780] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[15181.606731] R13: ffffffffa121b9c0 R14: ffffffffffffff10 R15: ffff8803162f6000
[15181.615677] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15181.625700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15181.633106] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15181.642080] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15181.651039] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15181.659986] Stack:
[15181.663213] ffff8800b2a3746c 0000000000000004 00000000ffffffff 0000000000000001
[15181.672504] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[15181.681803] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[15181.691097] Call Trace:
[15181.694837] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15181.704012] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15181.713176] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15181.722042] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15181.730708] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15181.737919] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15181.747284] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15181.753725] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15181.762021] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15181.769035] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15181.777315] Code: 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b <41> 5c 41 5d 41 5e 41 5f 5d c3 66 0f 1f 84 00 00 00 00 00 4c 8d
[15209.372305] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15209.381958] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15209.513903] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15209.527196] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15209.543449] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15209.552838] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[15209.565084] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[15209.572024] RAX: ffff8800b2a37468 RBX: ffff880703603d98 RCX: 0000000000000000
[15209.581007] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15209.589971] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15209.598922] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[15209.607875] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[15209.616828] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15209.626860] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15209.634259] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15209.643228] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15209.652188] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15209.661122] Stack:
[15209.664330] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15209.673597] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15209.682896] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15209.692193] Call Trace:
[15209.695915] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15209.705066] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15209.713925] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15209.722583] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15209.729782] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15209.739128] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15209.745547] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15209.753821] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15209.760829] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15209.769090] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[15224.004822] INFO: rcu_sched self-detected stall on CPU { 24} (t=7800150 jiffies g=907112 c=907111 q=24629680)
[15224.017296] Task dump for CPU 24:
[15224.022034] OI_scrub R running task 0 5551 2 0x00000088
[15224.031003] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[15224.040305] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[15224.049604] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[15224.058897] Call Trace:
[15224.062594] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[15224.070581] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[15224.077551] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[15224.085090] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[15224.092786] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[15224.101046] [<ffffffff81099697>] update_process_times+0x47/0x80
[15224.108602] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[15224.116622] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[15224.123752] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[15224.131356] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[15224.138641] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[15224.145522] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[15224.153549] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[15224.161366] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[15224.168786] <EOI> [<ffffffffa121cfff>] ? osd_inode_iteration+0x2af/0xcc0 [osd_ldiskfs]
[15224.178601] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15224.187677] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15224.196581] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15224.205188] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15224.213594] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15224.220552] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15224.229669] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15224.235836] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15224.243857] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15224.250613] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15249.373692] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15249.383143] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15249.512931] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15249.526119] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15249.542189] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15249.551489] RIP: 0010:[<ffffffffa1162b53>] [<ffffffffa1162b53>] ldiskfs_itable_unused_count+0x23/0x30 [ldiskfs]
[15249.563816] RSP: 0018:ffff880703603d08 EFLAGS: 00000297
[15249.570685] RAX: 0000000000000000 RBX: ffff880703603d57 RCX: ffff88042bd78800
[15249.579598] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[15249.588497] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15249.597410] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15249.606307] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[15249.615194] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15249.625153] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15249.632487] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15249.641380] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15249.650289] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15249.659191] Stack:
[15249.662357] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[15249.671612] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[15249.680860] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[15249.690115] Call Trace:
[15249.693821] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[15249.702952] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15249.712079] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15249.720916] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15249.729579] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15249.736755] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15249.746069] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15249.752458] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15249.760703] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15249.767680] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15249.775922] Code: 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76 07 0f b7 46 32 c1 e0 10 <09> d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 89 16
[15277.374668] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15277.384282] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15277.516259] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15277.529546] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15277.545793] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15277.555171] RIP: 0010:[<ffffffffa1217a29>] [<ffffffffa1217a29>] osd_scrub_next+0x9/0x3c0 [osd_ldiskfs]
[15277.566730] RSP: 0018:ffff880703603d08 EFLAGS: 00000287
[15277.573668] RAX: ffffffffa1217a20 RBX: 0000000000000001 RCX: ffff880703603d98
[15277.582635] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15277.591597] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[15277.600546] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15277.609481] R13: ffffea00027e3c80 R14: ffff880703603c70 R15: ffff880703603d08
[15277.618432] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15277.628454] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15277.635852] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15277.644810] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15277.653768] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15277.662717] Stack:
[15277.665946] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[15277.675252] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[15277.684553] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[15277.693844] Call Trace:
[15277.697574] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15277.706748] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15277.715912] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15277.724775] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15277.733450] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15277.740660] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15277.750004] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15277.756438] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15277.764708] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15277.771713] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15277.779991] Code: 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 <41> 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb
[15305.375647] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15305.385316] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15305.517425] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15305.530716] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15305.546981] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15305.556368] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[15305.568605] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[15305.575550] RAX: ffff88042bd78800 RBX: ffff880703603d98 RCX: ffff88042bd78800
[15305.584535] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[15305.593505] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15305.602466] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[15305.611421] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[15305.620375] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15305.630403] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15305.637809] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15305.646785] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15305.655747] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15305.664684] Stack:
[15305.667891] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15305.677160] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15305.686455] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15305.695737] Call Trace:
[15305.699461] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15305.708617] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15305.717488] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15305.726167] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15305.733372] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15305.742758] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15305.749197] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15305.757496] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15305.764509] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15305.772787] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[15333.376618] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15333.386257] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15333.518256] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15333.531558] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15333.547853] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15333.557252] RIP: 0010:[<ffffffffa121d1f0>] [<ffffffffa121d1f0>] osd_inode_iteration+0x4a0/0xcc0 [osd_ldiskfs]
[15333.569502] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[15333.576459] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[15333.585441] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15333.594425] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15333.603390] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15333.612340] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[15333.621286] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15333.631314] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15333.638715] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15333.647683] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15333.656645] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15333.665596] Stack:
[15333.668810] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15333.678094] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15333.687384] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15333.696688] Call Trace:
[15333.700413] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15333.709580] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15333.718453] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15333.727144] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15333.734375] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15333.743743] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15333.750176] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15333.758461] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15333.765468] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15333.773749] Code: 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c <8b> 85 58 ff ff ff 41 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b
[15361.377593] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15361.387244] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15361.519116] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15361.532433] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15361.548723] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15361.558122] RIP: 0010:[<ffffffffa121ba1d>] [<ffffffffa121ba1d>] osd_scrub_exec+0x8d/0x5b0 [osd_ldiskfs]
[15361.569815] RSP: 0018:ffff880703603ce8 EFLAGS: 00000292
[15361.576778] RAX: 0000000000000000 RBX: ffff880703603c78 RCX: 0000000000000000
[15361.585761] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15361.594737] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15361.603695] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[15361.612655] R13: ffffffffa121ba09 R14: ffffffffffffff10 R15: ffff8803162f6000
[15361.621596] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15361.631625] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15361.639028] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15361.647977] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15361.656934] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15361.665876] Stack:
[15361.669089] ffff8800b2a3746c 0000000000000004 00000000ffffffff 0000000000000001
[15361.678377] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[15361.687665] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[15361.696955] Call Trace:
[15361.700684] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15361.709848] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15361.719014] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15361.727881] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15361.736539] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15361.743740] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15361.753095] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15361.759526] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15361.767819] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15361.774825] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15361.783095] Code: 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b <41> 5c 41 5d 41 5e 41 5f 5d c3 66 0f 1f 84 00 00 00 00 00 4c 8d
[15389.378576] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15389.388210] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15389.520525] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15389.533818] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15389.550084] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15389.559475] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[15389.571716] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[15389.578658] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[15389.587643] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15389.596608] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15389.605567] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[15389.614522] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[15389.623475] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15389.633508] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15389.640913] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15389.649891] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15389.658852] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15389.667808] Stack:
[15389.671030] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15389.680330] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15389.689635] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15389.698932] Call Trace:
[15389.702660] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15389.711826] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15389.720691] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15389.729370] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15389.736576] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15389.745935] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15389.752368] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15389.760641] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15389.767659] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15389.775934] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[15404.014084] INFO: rcu_sched self-detected stall on CPU { 24} (t=7980153 jiffies g=907112 c=907111 q=25014902)
[15404.026520] Task dump for CPU 24:
[15404.031265] OI_scrub R running task 0 5551 2 0x00000088
[15404.040207] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[15404.049544] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[15404.058866] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[15404.068176] Call Trace:
[15404.071883] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[15404.079878] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[15404.086846] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[15404.094384] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[15404.102104] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[15404.110395] [<ffffffff81099697>] update_process_times+0x47/0x80
[15404.117965] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[15404.126006] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[15404.133155] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[15404.140762] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[15404.148070] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[15404.154965] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[15404.163013] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[15404.170858] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[15404.178295] <EOI> [<ffffffffa121ba79>] ? osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[15404.187530] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15404.196429] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15404.205338] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15404.213951] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15404.222368] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15404.229322] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15404.238407] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15404.244583] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15404.252607] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15404.259375] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15429.379961] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15429.389347] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15429.519050] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15429.532215] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15429.548280] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15429.557578] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[15429.569136] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[15429.576003] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[15429.584908] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15429.593805] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15429.602701] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15429.611600] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15429.620495] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15429.630463] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15429.637808] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15429.646715] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15429.655621] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15429.664521] Stack:
[15429.667689] ffff880703603df0 000000009f8f2001 ffffea00027e3c80 000000009f8f2001
[15429.676937] ffff880703603d57 ffff880703603d57 ffff8800b2a37468 0000000000000000
[15429.686188] ffff880703603d78 ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[15429.695437] Call Trace:
[15429.699116] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15429.708241] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15429.717370] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15429.726208] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15429.734841] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15429.742027] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15429.751336] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15429.757728] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15429.765973] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15429.772945] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15429.781179] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[15457.380943] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15457.390548] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15457.522794] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15457.536084] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15457.552356] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15457.561741] RIP: 0010:[<ffffffffa1162b36>] [<ffffffffa1162b36>] ldiskfs_itable_unused_count+0x6/0x30 [ldiskfs]
[15457.574079] RSP: 0018:ffff880703603d08 EFLAGS: 00000286
[15457.581024] RAX: ffff8800b2a37468 RBX: 0000000000000001 RCX: 0000000000000000
[15457.590009] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[15457.598977] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15457.607933] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15457.616885] R13: 0000000000000018 R14: ffff880703603c70 R15: 0000000000000202
[15457.625834] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15457.635857] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15457.643257] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15457.652218] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15457.661172] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15457.670103] Stack:
[15457.673306] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[15457.682594] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[15457.691880] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[15457.701157] Call Trace:
[15457.704898] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[15457.714254] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15457.723412] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15457.732271] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15457.740939] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15457.748149] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15457.757512] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15457.763952] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15457.772248] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15457.779260] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15457.787531] Code: 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 <48> 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76
[15485.381919] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15485.391564] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15485.523569] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15485.536845] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15485.553104] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15485.562498] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[15485.574750] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[15485.581698] RAX: ffff8800b2a37468 RBX: ffff880703603d98 RCX: 0000000000000000
[15485.590684] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15485.599655] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15485.608612] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[15485.617563] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[15485.626511] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15485.636537] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15485.643942] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15485.652909] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15485.661871] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15485.670825] Stack:
[15485.674031] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15485.683333] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15485.692639] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15485.701943] Call Trace:
[15485.705673] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15485.714842] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15485.723720] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15485.732398] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15485.739609] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15485.748970] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15485.755402] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15485.763698] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15485.770713] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15485.778995] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[15513.382894] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15513.392520] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15513.524687] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15513.537987] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15513.554279] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15513.563676] RIP: 0010:[<ffffffffa1217d73>] [<ffffffffa1217d73>] osd_scrub_next+0x353/0x3c0 [osd_ldiskfs]
[15513.575440] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[15513.582407] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[15513.591401] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15513.600380] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[15513.609342] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15513.618296] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15513.627245] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15513.637272] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15513.644682] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15513.653652] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15513.662626] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15513.671587] Stack:
[15513.674808] 000000009f8f2001 ffff880703603d08 ffff880703603d57 ffffea00027e3c80
[15513.684094] 000000009f8f2001 ffff880703603d57 ffff880703603d57 0000000000000000
[15513.693379] 0000000000000000 ffff880703603d78 ffff8800b2a36000 ffff8803162f6000
[15513.702647] Call Trace:
[15513.706366] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15513.715529] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15513.724686] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15513.733553] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15513.742228] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15513.749429] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15513.758780] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15513.765203] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15513.773487] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15513.780496] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15513.788766] Code: 8b 55 90 48 8b 4d 88 44 8b 85 74 ff ff ff e9 1d ff ff ff 48 89 d7 44 89 45 90 e8 f9 67 47 e0 44 8b 45 90 0f 1f 44 00 00 45 84 c0 <b8> 04 00 00 00 0f 85 eb fd ff ff 49 8d b7 6c 14 00 00 4c 89 e7
[15541.383870] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15541.393528] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15541.525682] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15541.538971] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15541.555205] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15541.564585] RIP: 0010:[<ffffffffa121d1e0>] [<ffffffffa121d1e0>] osd_inode_iteration+0x490/0xcc0 [osd_ldiskfs]
[15541.576832] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[15541.583760] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: 0000000000000000
[15541.592740] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15541.601702] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15541.610659] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[15541.619607] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[15541.628557] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15541.638585] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15541.645981] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15541.654958] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15541.663916] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15541.672858] Stack:
[15541.676063] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15541.685347] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15541.694637] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15541.703926] Call Trace:
[15541.707647] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15541.716817] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15541.725697] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15541.734376] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15541.741590] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15541.750967] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15541.757408] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15541.765710] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15541.772731] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15541.781013] Code: 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff <48> 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff
[15569.384853] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15569.394539] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15569.526559] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15569.539874] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15569.556173] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15569.565586] RIP: 0010:[<ffffffffa121ba1f>] [<ffffffffa121ba1f>] osd_scrub_exec+0x8f/0x5b0 [osd_ldiskfs]
[15569.577263] RSP: 0018:ffff880703603cf0 EFLAGS: 00000292
[15569.584227] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[15569.593224] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15569.602215] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15569.611189] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[15569.620146] R13: ffff880703603d08 R14: 0000000000000010 R15: ffffffffa121ba79
[15569.629098] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15569.639127] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15569.646536] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15569.655506] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15569.664471] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15569.673424] Stack:
[15569.676644] 0000000000000004 00000000ffffffff 0000000000000001 ffff880703603df0
[15569.685940] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[15569.695230] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[15569.704536] Call Trace:
[15569.708272] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15569.717443] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15569.726611] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15569.735486] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15569.744150] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15569.751352] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15569.760729] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15569.767162] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15569.775452] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15569.782466] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15569.790737] Code: 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c <41> 5d 41 5e 41 5f 5d c3 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0
[15584.023363] INFO: rcu_sched self-detected stall on CPU { 24} (t=8160156 jiffies g=907112 c=907111 q=25451775)
[15584.035815] Task dump for CPU 24:
[15584.040556] OI_scrub R running task 0 5551 2 0x00000088
[15584.049491] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[15584.058833] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[15584.068112] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[15584.077390] Call Trace:
[15584.081080] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[15584.089069] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[15584.096032] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[15584.103549] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[15584.111249] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[15584.119510] [<ffffffff81099697>] update_process_times+0x47/0x80
[15584.127074] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[15584.135081] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[15584.142216] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[15584.149814] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[15584.157091] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[15584.163973] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[15584.172003] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[15584.179849] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[15584.187275] <EOI> [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15584.196895] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15584.205803] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15584.214905] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15584.223822] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15584.232437] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15584.240834] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15584.247780] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15584.256884] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15584.263068] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15584.271087] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15584.277846] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15609.386240] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15609.395666] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15609.525368] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15609.538543] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15609.554580] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15609.563867] RIP: 0010:[<ffffffffa1217a65>] [<ffffffffa1217a65>] osd_scrub_next+0x45/0x3c0 [osd_ldiskfs]
[15609.575423] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[15609.582282] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[15609.591183] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15609.600088] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[15609.608992] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15609.617885] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15609.626785] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15609.636750] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15609.644100] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15609.653011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15609.661924] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15609.670825] Stack:
[15609.673993] 000000009f8f2001 ffffea00027e3c80 000000009f8f2001 0000000000000004
[15609.683247] ffff880703603d57 ffff880703603d57 0000000000000000 ffff880703603d78
[15609.692506] ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10 ffffffffa121b9a2
[15609.701757] Call Trace:
[15609.705456] [<ffffffffa121b9a2>] ? osd_scrub_exec+0x12/0x5b0 [osd_ldiskfs]
[15609.714205] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15609.723353] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15609.732500] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15609.741345] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15609.749991] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15609.757177] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15609.766500] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15609.772901] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15609.781149] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15609.788130] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15609.796366] Code: 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87 58 10 00 00 08 <0f> 84 62 03 00 00 4d 8d b7 e0 10 00 00 4d 3b b7 e0 10 00 00 0f
[15637.387209] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15637.396857] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15637.528569] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15637.541839] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15637.558070] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15637.567440] RIP: 0010:[<ffffffffa121cfb9>] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[15637.579677] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[15637.586620] RAX: 00000000000072ce RBX: 0000000000000001 RCX: ffff88042bd78800
[15637.595588] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[15637.604542] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15637.613500] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15637.622457] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[15637.631409] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15637.641435] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15637.648844] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15637.657807] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15637.666764] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15637.675715] Stack:
[15637.678933] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15637.688233] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15637.697525] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15637.706810] Call Trace:
[15637.710532] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15637.719697] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15637.728557] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15637.737224] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15637.744426] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15637.753777] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15637.760203] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15637.768478] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15637.775493] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15637.783767] Code: 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff <42> 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83
[15665.388183] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15665.397822] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15665.529995] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15665.543307] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15665.559588] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15665.568987] RIP: 0010:[<ffffffffa121ba09>] [<ffffffffa121ba09>] osd_scrub_exec+0x79/0x5b0 [osd_ldiskfs]
[15665.580657] RSP: 0018:ffff880703603c78 EFLAGS: 00000202
[15665.587618] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 00000000fb6db880
[15665.596605] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15665.605586] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15665.614544] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15665.623492] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15665.632437] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15665.642464] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15665.649876] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15665.658839] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15665.667808] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15665.676762] Stack:
[15665.679973] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 0000000000000001
[15665.689262] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[15665.698558] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[15665.707833] Call Trace:
[15665.711565] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15665.720728] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15665.729894] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15665.738760] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15665.747430] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15665.754640] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15665.763992] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15665.770423] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15665.778702] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15665.785705] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15665.793968] Code: 07 74 4f 41 83 f9 02 74 78 48 89 ca 44 89 c9 e8 0e c6 ff ff 85 c0 41 89 c1 0f 84 a3 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 <65> 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41
[15693.389164] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15693.398829] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15693.531039] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15693.544325] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15693.560578] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15693.569966] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[15693.582204] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[15693.589132] RAX: ffff88042bd78800 RBX: ffff880703603d98 RCX: ffff88042bd78800
[15693.598105] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[15693.607072] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15693.616032] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[15693.624984] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[15693.633932] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15693.643954] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15693.651356] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15693.660321] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15693.669271] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15693.678215] Stack:
[15693.681424] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15693.690686] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15693.699982] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15693.709237] Call Trace:
[15693.712954] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15693.722111] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15693.730980] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15693.739650] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15693.746854] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15693.756208] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15693.762646] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15693.770945] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15693.777956] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15693.786217] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[15721.390133] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15721.399817] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15721.531823] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15721.545142] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15721.561417] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15721.570813] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[15721.582477] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[15721.589427] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[15721.598411] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15721.607379] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[15721.616335] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c70
[15721.625273] R13: ffff880703603d08 R14: 0000000000000010 R15: ffffffffa1217d70
[15721.634207] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15721.644236] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15721.651633] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15721.660591] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15721.669549] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15721.678510] Stack:
[15721.681723] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[15721.691021] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[15721.700310] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[15721.709626] Call Trace:
[15721.713358] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15721.722530] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15721.731698] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15721.740574] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15721.749247] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15721.756465] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15721.765822] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15721.772255] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15721.780550] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15721.787572] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15721.795847] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[15749.391115] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15749.400768] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15749.533019] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15749.546326] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15749.562621] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15749.572015] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[15749.583685] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[15749.590633] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[15749.599615] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15749.608590] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15749.617554] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15749.626513] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15749.635474] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15749.645513] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15749.652930] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15749.661906] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15749.670872] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15749.679833] Stack:
[15749.683060] ffff880703603d08 0000000000000001 ffff880703603df0 000000009f8f2001
[15749.692376] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[15749.701682] 0000000020000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[15749.710978] Call Trace:
[15749.714725] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15749.723903] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15749.733061] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15749.741920] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15749.750592] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15749.757804] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15749.767184] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15749.773627] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15749.781923] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15749.788939] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15749.797222] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[15764.032626] INFO: rcu_sched self-detected stall on CPU { 24} (t=8340159 jiffies g=907112 c=907111 q=26089096)
[15764.045076] Task dump for CPU 24:
[15764.049809] OI_scrub R running task 0 5551 2 0x00000088
[15764.058739] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[15764.068061] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[15764.077364] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[15764.086638] Call Trace:
[15764.090319] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[15764.098303] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[15764.105265] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[15764.112781] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[15764.120472] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[15764.128726] [<ffffffff81099697>] update_process_times+0x47/0x80
[15764.136277] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[15764.144295] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[15764.151415] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[15764.159009] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[15764.166293] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[15764.173172] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[15764.181200] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[15764.189032] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[15764.196468] <EOI> [<ffffffffa121d1ca>] ? osd_inode_iteration+0x47a/0xcc0 [osd_ldiskfs]
[15764.206285] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15764.215391] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15764.224308] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15764.232931] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15764.241342] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15764.248295] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15764.257385] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15764.263550] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15764.271575] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15764.278340] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15789.392507] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15789.401894] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15789.531500] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15789.544673] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15789.560744] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15789.570032] RIP: 0010:[<ffffffffa1217a65>] [<ffffffffa1217a65>] osd_scrub_next+0x45/0x3c0 [osd_ldiskfs]
[15789.581606] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[15789.588463] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[15789.597373] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15789.606279] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[15789.615183] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15789.624091] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15789.632993] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15789.642967] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15789.650320] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15789.659242] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15789.668158] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15789.677064] Stack:
[15789.680240] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[15789.689489] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[15789.698731] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[15789.707980] Call Trace:
[15789.711676] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15789.720816] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15789.729936] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15789.738773] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15789.747418] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15789.754618] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15789.763937] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15789.770340] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15789.778586] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15789.785574] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15789.793824] Code: 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87 58 10 00 00 08 <0f> 84 62 03 00 00 4d 8d b7 e0 10 00 00 4d 3b b7 e0 10 00 00 0f
[15817.393478] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15817.403102] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15817.534833] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15817.548122] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15817.564380] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15817.573760] RIP: 0010:[<ffffffffa121d1d9>] [<ffffffffa121d1d9>] osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[15817.585989] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[15817.592923] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: 0000000000000000
[15817.601902] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15817.610871] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15817.619827] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15817.628774] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[15817.637723] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15817.647756] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15817.655159] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15817.664125] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15817.673075] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15817.682013] Stack:
[15817.685212] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15817.694486] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15817.703798] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15817.713085] Call Trace:
[15817.716795] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15817.725957] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15817.734827] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15817.743500] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15817.750704] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15817.760066] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15817.766496] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15817.774782] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15817.781793] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15817.790063] Code: 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5
[15845.394461] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15845.404130] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15845.536604] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15845.549924] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15845.566223] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15845.575626] RIP: 0010:[<ffffffffa121d00e>] [<ffffffffa121d00e>] osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[15845.587903] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[15845.594870] RAX: 0000000000000004 RBX: 0000000000000000 RCX: ffff880703603d98
[15845.603863] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[15845.612846] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[15845.621812] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[15845.630769] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[15845.639719] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15845.649742] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15845.657145] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15845.666107] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15845.675073] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15845.684020] Stack:
[15845.687238] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15845.696524] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15845.705807] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15845.715112] Call Trace:
[15845.718839] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15845.727998] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15845.736864] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15845.745543] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15845.752756] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15845.762117] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15845.768552] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15845.776836] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15845.783848] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15845.792119] Code: 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 <66> 90 0f 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90
[15873.395434] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15873.405058] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15873.536902] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15873.550202] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15873.566491] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15873.575886] RIP: 0010:[<ffffffffa121b9ac>] [<ffffffffa121b9ac>] osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[15873.587553] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[15873.594520] RAX: ffffffffa121b990 RBX: ffff880703603c37 RCX: 0000000000000000
[15873.603513] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15873.612499] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15873.621475] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15873.630425] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15873.639374] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15873.649395] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15873.656805] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15873.665779] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15873.674744] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15873.683696] Stack:
[15873.686916] ffff880703603d08 0000000000000001 ffff880703603df0 000000009f8f2001
[15873.696206] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[15873.705486] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[15873.714781] Call Trace:
[15873.718524] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15873.727692] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15873.736869] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15873.745743] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15873.754410] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15873.761617] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15873.770961] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15873.777388] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15873.785663] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15873.792679] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15873.800948] Code: 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 <48> 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48
[15901.396410] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[15901.406036] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15901.537684] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15901.550981] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15901.567232] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15901.576611] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[15901.588249] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[15901.595189] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[15901.604167] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15901.613135] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[15901.622089] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[15901.631040] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[15901.639992] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15901.650025] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15901.657433] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15901.666400] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15901.675359] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15901.684303] Stack:
[15901.687515] ffff880703603d08 0000000000000001 ffffea00027e3c80 000000009f8f2001
[15901.696795] ffff880703603d57 0000000000000001 ffffffffa1217a20 ffff880703603d98
[15901.706090] ffff880703603d78 ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10
[15901.715380] Call Trace:
[15901.719087] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15901.727959] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[15901.737126] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15901.746282] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15901.755147] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15901.763817] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15901.771024] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15901.780368] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15901.786799] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15901.795079] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15901.802078] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15901.810347] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[15929.397381] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15929.407029] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15929.538642] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15929.551905] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15929.568111] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15929.577472] RIP: 0010:[<ffffffffa121cfda>] [<ffffffffa121cfda>] osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[15929.589677] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[15929.596611] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[15929.605573] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[15929.614523] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[15929.623473] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[15929.632439] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[15929.641396] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15929.651426] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15929.658826] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15929.667800] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15929.676753] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15929.685695] Stack:
[15929.688915] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15929.698225] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15929.707504] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15929.716793] Call Trace:
[15929.720514] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15929.729683] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15929.738554] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15929.747227] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15929.754439] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15929.763802] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15929.770241] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15929.778535] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15929.785554] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15929.793834] Code: a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff <48> 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[15944.041896] INFO: rcu_sched self-detected stall on CPU { 24} (t=8520162 jiffies g=907112 c=907111 q=26370838)
[15944.054361] Task dump for CPU 24:
[15944.059123] OI_scrub R running task 0 5551 2 0x00000088
[15944.068091] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[15944.077436] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[15944.086756] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[15944.096058] Call Trace:
[15944.099772] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[15944.107806] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[15944.114804] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[15944.122367] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[15944.130087] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[15944.138390] [<ffffffff81099697>] update_process_times+0x47/0x80
[15944.145971] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[15944.154011] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[15944.161177] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[15944.168793] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[15944.176102] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[15944.183002] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[15944.191063] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[15944.198907] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[15944.206349] <EOI> [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15944.215700] [<ffffffffa1217a3b>] ? osd_scrub_next+0x1b/0x3c0 [osd_ldiskfs]
[15944.224227] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[15944.233141] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15944.242047] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15944.250661] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15944.259077] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15944.266027] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15944.275131] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15944.281303] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15944.289337] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15944.296102] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15969.398773] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15969.408190] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15969.537894] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15969.551065] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15969.567103] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15969.576384] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[15969.588518] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[15969.595374] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[15969.604276] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[15969.613172] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[15969.622069] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[15969.630965] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[15969.639862] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15969.649829] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15969.657178] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15969.666091] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15969.675002] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15969.683904] Stack:
[15969.687069] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15969.696319] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15969.705575] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15969.714838] Call Trace:
[15969.718534] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15969.727675] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15969.736518] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15969.745172] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15969.752355] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15969.761670] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15969.768075] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15969.776330] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15969.783312] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15969.791563] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[15997.399754] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[15997.409367] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[15997.541765] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[15997.555079] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[15997.571374] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[15997.580783] RIP: 0010:[<ffffffffa121cfe9>] [<ffffffffa121cfe9>] osd_inode_iteration+0x299/0xcc0 [osd_ldiskfs]
[15997.593042] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[15997.600009] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff880703603d98
[15997.609005] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff88042bd79800
[15997.617998] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[15997.626968] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[15997.635926] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffffea00027e3c80
[15997.644889] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[15997.654924] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[15997.662337] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[15997.671309] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[15997.680283] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[15997.689237] Stack:
[15997.692462] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[15997.701753] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[15997.711044] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[15997.720351] Call Trace:
[15997.724094] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[15997.733266] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[15997.742149] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[15997.750843] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[15997.758062] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[15997.767432] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[15997.773877] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15997.782169] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[15997.789189] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[15997.797475] Code: 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89
[16025.400730] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16025.410363] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16025.542469] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16025.555785] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16025.572095] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16025.581509] RIP: 0010:[<ffffffffa121cffc>] [<ffffffffa121cffc>] osd_inode_iteration+0x2ac/0xcc0 [osd_ldiskfs]
[16025.593767] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[16025.600747] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[16025.609747] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[16025.618743] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[16025.627712] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[16025.636669] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[16025.645619] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16025.655646] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16025.663047] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16025.672014] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16025.680978] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16025.689927] Stack:
[16025.693134] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16025.702426] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16025.711714] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16025.721010] Call Trace:
[16025.724723] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16025.733883] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16025.742755] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16025.751437] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16025.758642] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16025.767999] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16025.774429] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16025.782696] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16025.789714] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16025.797985] Code: 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 <41> 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 0f
[16053.401699] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16053.411337] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16053.543296] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16053.556597] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16053.572890] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16053.582297] RIP: 0010:[<ffffffffa1217b69>] [<ffffffffa1217b69>] osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[16053.594067] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[16053.601028] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[16053.610016] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16053.618995] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16053.627956] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16053.636910] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16053.645860] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16053.655886] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16053.663298] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16053.672263] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16053.681233] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16053.690189] Stack:
[16053.693408] 000000009f8f2001 0000000000000004 ffff880703603d57 ffff880703603df0
[16053.702705] 0000000000000000 ffff880703603d57 000000009f8f2001 0000000000000004
[16053.711993] ffff880703603d57 0000000000000000 0000000000000000 ffff880703603d78
[16053.721295] Call Trace:
[16053.725039] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16053.734216] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16053.743390] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16053.752271] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16053.760950] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16053.768168] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16053.777529] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16053.783959] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16053.792251] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16053.799272] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16053.807556] Code: 45 90 74 49 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 <48> 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83
[16078.645768] Lustre: soaked-MDT0000: Client e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100) reconnecting
[16078.660509] Lustre: soaked-MDT0000: Connection restored to e211cd09-6b0c-e43b-059c-f657101042a0 (at 192.168.1.117@o2ib100)
[16081.402676] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16081.412372] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16081.544210] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16081.557492] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16081.573751] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16081.583139] RIP: 0010:[<ffffffffa1162b46>] [<ffffffffa1162b46>] ldiskfs_itable_unused_count+0x16/0x30 [ldiskfs]
[16081.595579] RSP: 0018:ffff880703603d08 EFLAGS: 00000246
[16081.602542] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[16081.611543] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[16081.620540] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[16081.629532] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16081.638530] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[16081.647523] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16081.657582] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16081.665014] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16081.674014] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16081.682993] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16081.691966] Stack:
[16081.695225] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[16081.704545] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[16081.713867] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[16081.723171] Call Trace:
[16081.726926] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[16081.736115] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16081.745308] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16081.754199] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16081.762893] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16081.770136] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16081.779505] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16081.785963] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16081.794267] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16081.801305] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16081.809612] Code: 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 <48> 83 39 3f 76 07 0f b7 46 32 c1 e0 10 09 d0 5d c3 66 0f 1f 84
[16109.403650] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16109.413366] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16109.545151] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16109.558432] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16109.574685] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16109.584061] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[16109.596289] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[16109.603215] RAX: ffff8800b2a37468 RBX: 0000000000000000 RCX: 0000000000000000
[16109.612189] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16109.621147] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16109.630090] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[16109.639038] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[16109.647990] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16109.658014] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16109.665410] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16109.674376] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16109.683331] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16109.692272] Stack:
[16109.695499] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16109.704787] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16109.714059] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16109.723367] Call Trace:
[16109.727082] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16109.736251] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16109.745127] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16109.753793] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16109.760996] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16109.770366] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16109.776808] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16109.785084] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16109.792102] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16109.800373] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[16124.051161] INFO: rcu_sched self-detected stall on CPU { 24} (t=8700165 jiffies g=907112 c=907111 q=26715744)
[16124.063636] Task dump for CPU 24:
[16124.068400] OI_scrub R running task 0 5551 2 0x00000088
[16124.077348] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[16124.086690] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[16124.096027] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[16124.105316] Call Trace:
[16124.109024] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[16124.117022] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[16124.124001] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[16124.131543] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[16124.139262] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[16124.147546] [<ffffffff81099697>] update_process_times+0x47/0x80
[16124.155126] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[16124.163162] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[16124.170311] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[16124.177916] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[16124.185216] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[16124.192097] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[16124.200145] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[16124.207971] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[16124.215394] <EOI> [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16124.225205] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16124.234300] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16124.243205] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16124.251816] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16124.260227] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16124.267182] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16124.276276] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16124.282445] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16124.290464] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16124.297224] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16149.405043] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16149.414385] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16149.544062] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16149.557241] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16149.573310] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16149.582605] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[16149.594766] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[16149.601636] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[16149.610551] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[16149.619450] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16149.628360] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[16149.637263] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[16149.646157] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16149.656122] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16149.663477] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16149.672391] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16149.681304] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16149.690204] Stack:
[16149.693379] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16149.702644] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16149.711904] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16149.721162] Call Trace:
[16149.724859] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16149.734002] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16149.742847] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16149.751510] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16149.758710] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16149.768044] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16149.774456] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16149.782725] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16149.789711] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16149.797957] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[16177.406024] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16177.415667] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16177.547621] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16177.560949] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16177.577247] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16177.586659] RIP: 0010:[<ffffffffa121ba64>] [<ffffffffa121ba64>] osd_scrub_exec+0xd4/0x5b0 [osd_ldiskfs]
[16177.598340] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[16177.605302] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[16177.614289] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16177.623267] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[16177.632229] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16177.641181] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16177.650140] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16177.660173] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16177.667582] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16177.676554] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16177.685514] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16177.694462] Stack:
[16177.697689] ffff880703603d08 ffff880703603d57 ffff880703603df0 000000009f8f2001
[16177.706978] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[16177.716272] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[16177.725583] Call Trace:
[16177.729306] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16177.738472] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16177.747625] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16177.756495] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16177.765169] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16177.772380] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16177.781737] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16177.788169] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16177.796452] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16177.803460] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16177.811734] Code: 4d 88 4c 89 ef e8 1d fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 <f6> 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00
[16205.407000] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16205.416638] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16205.548557] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16205.561843] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16205.578099] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16205.587495] RIP: 0010:[<ffffffffa1217a29>] [<ffffffffa1217a29>] osd_scrub_next+0x9/0x3c0 [osd_ldiskfs]
[16205.599058] RSP: 0018:ffff880703603d08 EFLAGS: 00000287
[16205.606004] RAX: ffffffffa1217a20 RBX: ffff880703603d57 RCX: ffff880703603d98
[16205.614986] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16205.623948] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16205.632897] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16205.641848] R13: ffffea00027e3c80 R14: ffff880703603c78 R15: ffff880703603d08
[16205.650803] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16205.660835] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16205.668239] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16205.677209] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16205.686158] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16205.695108] Stack:
[16205.698311] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[16205.707589] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[16205.716891] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[16205.726183] Call Trace:
[16205.729918] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16205.739095] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16205.748253] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16205.757106] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16205.765774] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16205.772986] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16205.782349] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16205.788785] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16205.797068] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16205.804083] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16205.812356] Code: 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 <41> 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb
[16233.407970] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16233.417671] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16233.549536] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16233.562822] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16233.579073] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16233.588456] RIP: 0010:[<ffffffffa121cfda>] [<ffffffffa121cfda>] osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[16233.600689] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[16233.607635] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[16233.616613] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[16233.625579] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[16233.634538] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[16233.643492] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[16233.652451] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16233.662481] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16233.669898] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16233.678874] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16233.687829] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16233.696774] Stack:
[16233.700001] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16233.709292] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16233.718597] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16233.727874] Call Trace:
[16233.731602] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16233.740759] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16233.749632] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16233.758317] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16233.765523] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16233.774899] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16233.781334] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16233.789624] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16233.796637] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16233.804919] Code: a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff <48> 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[16261.408947] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16261.418621] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16261.550701] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16261.564014] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16261.580316] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16261.589732] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[16261.601993] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[16261.608965] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[16261.617959] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[16261.626944] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16261.635914] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[16261.644869] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffffea00027e3c80
[16261.653811] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16261.663840] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16261.671247] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16261.680215] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16261.689181] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16261.698138] Stack:
[16261.701353] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16261.710641] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16261.719929] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16261.729242] Call Trace:
[16261.732970] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16261.742128] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16261.751004] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16261.759696] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16261.766918] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16261.776267] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16261.782707] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16261.790989] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16261.798002] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16261.806283] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[16289.409920] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16289.419608] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16289.551684] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16289.565003] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16289.581318] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16289.590728] RIP: 0010:[<ffffffffa121b999>] [<ffffffffa121b999>] osd_scrub_exec+0x9/0x5b0 [osd_ldiskfs]
[16289.602313] RSP: 0018:ffff880703603d08 EFLAGS: 00000293
[16289.609274] RAX: ffffffffa121b990 RBX: ffff880703603d57 RCX: 0000000000000000
[16289.618261] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16289.627238] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[16289.636198] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16289.645149] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[16289.654098] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16289.664114] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16289.671517] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16289.680476] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16289.689429] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16289.698368] Stack:
[16289.701585] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[16289.710869] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[16289.720162] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[16289.729436] Call Trace:
[16289.733160] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16289.742320] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16289.751472] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16289.760342] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16289.769011] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16289.776207] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16289.785552] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16289.791970] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16289.800253] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16289.807255] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16289.815520] Code: 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 <41> 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b
[16304.060431] INFO: rcu_sched self-detected stall on CPU { 24} (t=8880168 jiffies g=907112 c=907111 q=27253292)
[16304.072887] Task dump for CPU 24:
[16304.077600] OI_scrub R running task 0 5551 2 0x00000088
[16304.086510] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[16304.095835] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[16304.105127] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[16304.114399] Call Trace:
[16304.118071] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[16304.126042] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[16304.133001] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[16304.140508] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[16304.148189] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[16304.156430] [<ffffffff81099697>] update_process_times+0x47/0x80
[16304.163968] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[16304.171977] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[16304.179086] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[16304.186661] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[16304.193928] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[16304.200798] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[16304.208813] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[16304.216623] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[16304.224047] <EOI> [<ffffffffa1217a65>] ? osd_scrub_next+0x45/0x3c0 [osd_ldiskfs]
[16304.233281] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16304.242175] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16304.251067] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16304.259672] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16304.268082] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16304.275030] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16304.284110] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16304.290280] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16304.298296] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16304.305044] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16329.411321] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16329.420737] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16329.550475] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16329.563643] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16329.579713] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16329.589001] RIP: 0010:[<ffffffffa121cfb4>] [<ffffffffa121cfb4>] osd_inode_iteration+0x264/0xcc0 [osd_ldiskfs]
[16329.601145] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[16329.608006] RAX: ffff8800b2a37468 RBX: 0000000000000001 RCX: 0000000000000000
[16329.616912] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[16329.625806] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16329.634706] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16329.643614] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[16329.652510] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16329.662474] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16329.669818] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16329.678719] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16329.687621] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16329.696530] Stack:
[16329.699699] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16329.708934] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16329.718174] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16329.727414] Call Trace:
[16329.731099] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16329.740233] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16329.749071] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16329.757724] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16329.764920] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16329.774277] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16329.780683] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16329.788955] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16329.795944] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16329.804202] Code: 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 <e8> 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48
[16357.412289] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16357.421950] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16357.553926] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16357.567258] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16357.583562] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16357.592969] RIP: 0010:[<ffffffffa121d1c3>] [<ffffffffa121d1c3>] osd_inode_iteration+0x473/0xcc0 [osd_ldiskfs]
[16357.605222] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[16357.612197] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: ffff880703603d98
[16357.621191] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[16357.630173] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[16357.639153] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[16357.648116] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffffea00027e3c80
[16357.657076] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16357.667104] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16357.674515] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16357.683491] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16357.692456] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16357.701403] Stack:
[16357.704632] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16357.713921] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16357.723224] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16357.732534] Call Trace:
[16357.736258] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16357.745431] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16357.754307] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16357.762985] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16357.770200] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16357.779548] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16357.785978] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16357.794256] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16357.801269] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16357.809541] Code: 00 00 48 c7 05 9b 86 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 <4c> 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff
[16385.413265] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16385.422915] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16385.554787] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16385.568106] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16385.584411] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16385.593817] RIP: 0010:[<ffffffffa1217a2b>] [<ffffffffa1217a2b>] osd_scrub_next+0xb/0x3c0 [osd_ldiskfs]
[16385.605408] RSP: 0018:ffff880703603d00 EFLAGS: 00000287
[16385.612380] RAX: ffffffffa1217a20 RBX: 0000000000000001 RCX: ffff880703603d98
[16385.621365] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16385.630351] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16385.639316] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16385.648268] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[16385.657211] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16385.667242] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16385.674659] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16385.683623] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16385.692569] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16385.701507] Stack:
[16385.704712] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[16385.713988] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[16385.723269] ffff8800b2a37468 0000000020000000 010000000000000c 0000000000000000
[16385.732549] Call Trace:
[16385.736260] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16385.745431] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16385.754588] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16385.763463] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16385.772126] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16385.779325] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16385.788663] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16385.795094] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16385.803364] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16385.810374] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16385.818634] Code: 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 <49> 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83
[16388.169870] Lustre: soaked-MDT0000: Client 40e8ebdf-e84f-0dfe-7f82-ddc39c67a41e (at 192.168.1.132@o2ib100) reconnecting
[16388.184482] Lustre: soaked-MDT0000: Connection restored to 7eb6195d-0392-6b9e-dc88-22a505b06cf8 (at 192.168.1.132@o2ib100)
[16413.414240] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16413.423911] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16413.555680] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16413.568962] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16413.585201] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16413.594577] RIP: 0010:[<ffffffffa1217a65>] [<ffffffffa1217a65>] osd_scrub_next+0x45/0x3c0 [osd_ldiskfs]
[16413.606231] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[16413.613179] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[16413.622160] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16413.631130] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16413.640107] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16413.649088] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16413.658085] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16413.668146] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16413.675590] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16413.684582] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16413.693571] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16413.702551] Stack:
[16413.705815] 000000009f8f2001 ffff880703603d08 0000000000000001 ffff880703603df0
[16413.715141] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[16413.724462] 0000000000000001 0000000000000004 ffff880703603d98 ffff880703603d78
[16413.733787] Call Trace:
[16413.737541] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16413.746738] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16413.755917] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16413.764808] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16413.773504] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16413.780744] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16413.790137] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16413.796595] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16413.804915] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16413.811952] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16413.820257] Code: 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87 58 10 00 00 08 <0f> 84 62 03 00 00 4d 8d b7 e0 10 00 00 4d 3b b7 e0 10 00 00 0f
[16441.415227] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16441.424911] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16441.557107] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16441.570405] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16441.586668] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16441.596056] RIP: 0010:[<ffffffffa121cfe9>] [<ffffffffa121cfe9>] osd_inode_iteration+0x299/0xcc0 [osd_ldiskfs]
[16441.608310] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[16441.615256] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff880703603d98
[16441.624239] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff88042bd79800
[16441.633210] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[16441.642168] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[16441.651125] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[16441.660085] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16441.670122] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16441.677537] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16441.686510] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16441.695478] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16441.704443] Stack:
[16441.707666] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16441.716962] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16441.726260] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16441.735555] Call Trace:
[16441.739279] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16441.748455] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16441.757338] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16441.766028] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16441.773238] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16441.782588] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16441.789037] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16441.797334] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16441.804367] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16441.812652] Code: 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89
[16469.416191] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16469.425860] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16469.557882] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16469.571195] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16469.587507] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16469.596922] RIP: 0010:[<ffffffffa1217d73>] [<ffffffffa1217d73>] osd_scrub_next+0x353/0x3c0 [osd_ldiskfs]
[16469.608706] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[16469.615672] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[16469.624666] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16469.633659] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16469.642633] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16469.651591] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16469.660552] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16469.670584] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16469.677997] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16469.686975] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16469.695948] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16469.704898] Stack:
[16469.708114] 000000009f8f2001 ffff880703603df0 ffff880703603d57 ffff880703603df0
[16469.717415] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[16469.726711] 0000000000000001 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[16469.736000] Call Trace:
[16469.739722] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16469.748893] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16469.758052] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16469.766919] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16469.775581] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16469.782790] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16469.792132] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16469.798555] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16469.806826] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16469.813829] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16469.822107] Code: 8b 55 90 48 8b 4d 88 44 8b 85 74 ff ff ff e9 1d ff ff ff 48 89 d7 44 89 45 90 e8 f9 67 47 e0 44 8b 45 90 0f 1f 44 00 00 45 84 c0 <b8> 04 00 00 00 0f 85 eb fd ff ff 49 8d b7 6c 14 00 00 4c 89 e7
[16484.069702] INFO: rcu_sched self-detected stall on CPU { 24} (t=9060171 jiffies g=907112 c=907111 q=28308206)
[16484.082158] Task dump for CPU 24:
[16484.086888] OI_scrub R running task 0 5551 2 0x00000088
[16484.095824] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[16484.105139] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[16484.114438] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[16484.123708] Call Trace:
[16484.127384] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[16484.135357] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[16484.142305] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[16484.149818] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[16484.157509] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[16484.165767] [<ffffffff81099697>] update_process_times+0x47/0x80
[16484.173315] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[16484.181326] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[16484.188437] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[16484.196018] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[16484.203299] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[16484.210166] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[16484.218185] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[16484.226004] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[16484.233421] <EOI> [<ffffffffa121cfa1>] ? osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[16484.243228] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16484.252322] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16484.261220] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16484.269832] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16484.278251] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16484.285205] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16484.294295] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16484.300461] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16484.308481] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16484.315235] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16509.417584] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16509.427000] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16509.556822] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16509.569994] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16509.586063] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16509.595371] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[16509.607511] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[16509.614372] RAX: ffff88042bd78800 RBX: ffff880703603d57 RCX: ffff88042bd78800
[16509.623279] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[16509.632183] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16509.641093] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16509.650001] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[16509.658900] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16509.668879] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16509.676230] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16509.685146] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16509.694077] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16509.702979] Stack:
[16509.706151] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16509.715408] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16509.724664] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16509.733923] Call Trace:
[16509.737620] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16509.746759] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16509.755609] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16509.764259] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16509.771457] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16509.780800] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16509.787210] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16509.795464] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16509.802448] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16509.810709] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[16537.418566] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16537.428194] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16537.560709] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16537.574034] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16537.590347] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16537.599776] RIP: 0010:[<ffffffffa121b9cd>] [<ffffffffa121b9cd>] osd_scrub_exec+0x3d/0x5b0 [osd_ldiskfs]
[16537.611468] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[16537.618441] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[16537.627440] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16537.636428] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[16537.645401] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16537.654364] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16537.663322] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16537.673364] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16537.680771] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16537.689746] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16537.698721] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16537.707675] Stack:
[16537.710896] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 0000000000000001
[16537.720198] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[16537.729497] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[16537.738794] Call Trace:
[16537.742536] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16537.751716] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16537.760880] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16537.769754] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16537.778432] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16537.785645] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16537.795024] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16537.801461] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16537.809749] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16537.816760] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16537.825031] Code: 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8d 43 08 48 85 db <4c> 0f 45 e0 41 83 f9 04 0f 84 84 00 00 00 41 83 f9 07 74 4f 41
[16565.419541] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16565.429187] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16565.561314] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16565.574598] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16565.590838] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16565.600225] RIP: 0010:[<ffffffffa121d1ca>] [<ffffffffa121d1ca>] osd_inode_iteration+0x47a/0xcc0 [osd_ldiskfs]
[16565.612465] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[16565.619414] RAX: 0000000000000004 RBX: ffff880703603d57 RCX: ffff880703603d98
[16565.628395] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[16565.637357] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16565.646317] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16565.655270] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[16565.664229] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16565.674266] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16565.681687] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16565.690668] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16565.699625] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16565.708573] Stack:
[16565.711789] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16565.721082] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16565.730392] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16565.739696] Call Trace:
[16565.743426] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16565.752598] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16565.761472] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16565.770159] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16565.777376] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16565.786743] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16565.793192] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16565.801496] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16565.808520] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16565.816798] Code: 02 00 00 00 00 00 c7 05 89 86 02 00 01 00 00 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff <48> 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[16593.420520] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16593.430150] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16593.562221] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16593.575539] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16593.591845] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16593.601268] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[16593.613595] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[16593.620563] RAX: ffff8800b2a37468 RBX: ffff880703603d98 RCX: 0000000000000000
[16593.629602] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16593.638588] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16593.647558] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[16593.656513] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[16593.665497] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16593.675552] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16593.682953] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16593.691930] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16593.700886] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16593.709831] Stack:
[16593.713037] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16593.722376] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16593.731661] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16593.740952] Call Trace:
[16593.744670] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16593.753890] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16593.762763] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16593.771477] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16593.778684] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16593.788026] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16593.794562] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16593.802841] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16593.809847] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16593.818120] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[16621.421486] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16621.431107] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16621.563027] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16621.576326] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16621.592607] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16621.602003] RIP: 0010:[<ffffffffa1217b69>] [<ffffffffa1217b69>] osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[16621.613762] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[16621.620714] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[16621.629691] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16621.638664] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16621.647609] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16621.656555] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16621.665505] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16621.675523] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16621.682923] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16621.691891] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16621.700853] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16621.709814] Stack:
[16621.713044] 000000009f8f2001 0000000000000004 0000000000000001 ffff880703603df0
[16621.722334] ffff880703603d98 ffff880703603d57 000000009f8f2001 0000000000000004
[16621.731635] ffff880703603d57 ffff8800b2a37468 0000000000000000 ffff880703603d78
[16621.740934] Call Trace:
[16621.744665] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16621.753831] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16621.763000] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16621.771869] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16621.780545] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16621.787753] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16621.797099] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16621.803525] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16621.811801] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16621.818800] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16621.827073] Code: 45 90 74 49 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 <48> 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83
[16649.422461] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16649.432106] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16649.563936] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16649.577258] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16649.593508] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16649.602894] RIP: 0010:[<ffffffffa1217b69>] [<ffffffffa1217b69>] osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[16649.614644] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[16649.621583] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[16649.630568] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16649.639534] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16649.648489] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16649.657438] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16649.666389] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16649.676417] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16649.683824] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16649.692790] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16649.701749] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16649.710692] Stack:
[16649.713900] 000000009f8f2001 0000000000000004 0000000000000001 ffffffffa1217a20
[16649.723195] ffff880703603d98 ffff880703603d57 ffff8800b2a36000 ffff8803162f6000
[16649.732509] ffffffffffffff10 ffffffffa1217a3e 0000000000000010 0000000000000287
[16649.741823] Call Trace:
[16649.745549] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16649.754434] [<ffffffffa1217a3e>] ? osd_scrub_next+0x1e/0x3c0 [osd_ldiskfs]
[16649.763223] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16649.772388] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16649.781544] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16649.790438] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16649.799118] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16649.806334] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16649.815720] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16649.822155] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16649.830437] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16649.837459] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16649.845736] Code: 45 90 74 49 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 <48> 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83
[16664.078993] INFO: rcu_sched self-detected stall on CPU { 24} (t=9240174 jiffies g=907112 c=907111 q=29961699)
[16664.091454] Task dump for CPU 24:
[16664.096167] OI_scrub R running task 0 5551 2 0x00000088
[16664.105109] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[16664.114414] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[16664.123689] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[16664.132922] Call Trace:
[16664.136573] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[16664.144542] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[16664.151482] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[16664.158982] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[16664.166649] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[16664.174883] [<ffffffff81099697>] update_process_times+0x47/0x80
[16664.182407] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[16664.190398] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[16664.197501] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[16664.205071] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[16664.212338] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[16664.219206] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[16664.227229] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[16664.235067] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[16664.242513] <EOI> [<ffffffffa121b9c4>] ? osd_scrub_exec+0x34/0x5b0 [osd_ldiskfs]
[16664.251753] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16664.260661] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16664.269566] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16664.278173] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16664.286577] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16664.293522] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16664.302626] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16664.308803] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16664.316839] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16664.323591] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16689.423869] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16689.433269] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16689.562901] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16689.576069] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16689.592125] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16689.601415] RIP: 0010:[<ffffffffa121b990>] [<ffffffffa121b990>] osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16689.613367] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[16689.620213] RAX: ffffffffa121b990 RBX: ffff880703603d98 RCX: 0000000000000000
[16689.629113] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16689.638004] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16689.646900] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[16689.655795] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[16689.664670] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16689.674630] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16689.681966] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16689.690864] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16689.699765] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16689.708649] Stack:
[16689.711809] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[16689.721055] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[16689.730291] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[16689.739535] Call Trace:
[16689.743212] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16689.752542] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16689.761678] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16689.770514] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16689.779156] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16689.786332] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16689.795678] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16689.802085] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16689.810351] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16689.817329] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16689.825571] Code: b5 07 00 00 c7 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45
[16717.424831] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16717.434464] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16717.566203] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16717.579498] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16717.595748] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16717.605140] RIP: 0010:[<ffffffffa121ba25>] [<ffffffffa121ba25>] osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[16717.616787] RSP: 0018:ffff880703603d08 EFLAGS: 00000292
[16717.623728] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000000
[16717.632704] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16717.641675] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[16717.650633] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16717.659587] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[16717.668530] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16717.678555] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16717.685953] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16717.694915] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16717.703883] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16717.712826] Stack:
[16717.716038] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[16717.725324] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[16717.734617] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[16717.743926] Call Trace:
[16717.747644] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16717.756807] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16717.765962] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16717.774823] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16717.783483] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16717.790681] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16717.800024] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16717.806445] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16717.814715] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16717.821717] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16717.829984] Code: 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d
[16745.425806] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16745.435489] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16745.567226] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16745.580507] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16745.596738] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16745.606114] RIP: 0010:[<ffffffffa1217a29>] [<ffffffffa1217a29>] osd_scrub_next+0x9/0x3c0 [osd_ldiskfs]
[16745.617656] RSP: 0018:ffff880703603d08 EFLAGS: 00000287
[16745.624587] RAX: ffffffffa1217a20 RBX: 0000000000000001 RCX: ffff880703603d98
[16745.633560] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16745.642506] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16745.651458] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16745.660396] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000296
[16745.669322] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16745.679333] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16745.686722] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16745.695675] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16745.704621] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16745.713553] Stack:
[16745.716750] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[16745.726046] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[16745.735340] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[16745.744627] Call Trace:
[16745.748345] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16745.757511] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16745.766669] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16745.775521] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16745.784179] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16745.791375] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16745.800733] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16745.807148] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16745.815436] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16745.822436] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16745.830702] Code: 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 <41> 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb
[16773.426782] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16773.436507] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16773.568363] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16773.581647] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16773.597885] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16773.607269] RIP: 0010:[<ffffffffa1217b80>] [<ffffffffa1217b80>] osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[16773.619035] RSP: 0018:ffff880703603ce0 EFLAGS: 00000282
[16773.625967] RAX: 0000000000000004 RBX: ffff880703603c70 RCX: ffff880703603d98
[16773.634942] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[16773.643900] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16773.652847] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[16773.661774] R13: ffffffffa1217d70 R14: ffffffffffffff10 R15: ffff8803162f6000
[16773.670713] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16773.680730] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16773.688127] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16773.697085] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16773.706044] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16773.714990] Stack:
[16773.718196] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[16773.727504] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[16773.736801] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[16773.746097] Call Trace:
[16773.749818] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16773.758989] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16773.768149] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16773.777019] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16773.785679] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16773.792900] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16773.802255] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16773.808685] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16773.816972] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16773.823990] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16773.832260] Code: 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0
[16801.427757] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16801.437414] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16801.569357] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16801.582646] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16801.598908] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16801.608298] RIP: 0010:[<ffffffffa121d1d9>] [<ffffffffa121d1d9>] osd_inode_iteration+0x489/0xcc0 [osd_ldiskfs]
[16801.620543] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[16801.627518] RAX: 0000000000000004 RBX: 000000009f8f2001 RCX: 0000000000000000
[16801.636496] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16801.645462] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16801.654417] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[16801.663365] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[16801.672318] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16801.682346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16801.689754] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16801.698727] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16801.707691] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16801.716627] Stack:
[16801.719837] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16801.729123] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16801.738446] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16801.747744] Call Trace:
[16801.751497] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16801.760660] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16801.769536] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16801.778226] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16801.785430] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16801.794782] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16801.801215] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16801.809498] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16801.816511] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16801.824782] Code: 00 e8 7c eb 97 ff e9 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5
[16814.825524] Lustre: soaked-MDT0000: Client 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100) reconnecting
[16814.838837] Lustre: soaked-MDT0000: Connection restored to 91eacee7-b83c-19ec-9326-47c5ebda930d (at 192.168.1.116@o2ib100)
[16829.428739] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16829.438412] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16829.570440] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16829.583752] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16829.600039] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16829.609449] RIP: 0010:[<ffffffffa1217b69>] [<ffffffffa1217b69>] osd_scrub_next+0x149/0x3c0 [osd_ldiskfs]
[16829.621202] RSP: 0018:ffff880703603c70 EFLAGS: 00000202
[16829.628168] RAX: 0000000000000004 RBX: ffff880703603c37 RCX: ffff880703603d98
[16829.637166] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16829.646142] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16829.655130] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16829.664111] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16829.673103] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16829.683166] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16829.690609] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16829.699609] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16829.708595] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16829.717587] Stack:
[16829.720836] 000000009f8f2001 0000000000000004 ffff880703603d57 ffff880703603df0
[16829.730155] 0000000000000000 ffff880703603d57 000000009f8f2001 0000000000000004
[16829.739498] ffff880703603d57 ffff88042bd78800 ffff88042bd78800 00000000000072cf
[16829.748836] Call Trace:
[16829.752593] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16829.761804] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16829.770998] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16829.779895] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16829.788602] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16829.795847] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16829.805245] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16829.811711] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16829.820020] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16829.827065] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16829.835377] Code: 45 90 74 49 49 8d 9f 10 11 00 00 48 89 df e8 df 6a 47 e0 41 c7 87 58 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 <48> 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83
[16844.088243] INFO: rcu_sched self-detected stall on CPU { 24} (t=9420177 jiffies g=907112 c=907111 q=31174035)
[16844.100730] Task dump for CPU 24:
[16844.105467] OI_scrub R running task 0 5551 2 0x00000088
[16844.114404] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[16844.123733] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[16844.133031] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[16844.142313] Call Trace:
[16844.145998] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[16844.153981] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[16844.160947] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[16844.168468] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[16844.176169] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[16844.184430] [<ffffffff81099697>] update_process_times+0x47/0x80
[16844.191969] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[16844.199984] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[16844.207102] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[16844.214696] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[16844.221974] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[16844.228853] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[16844.236873] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[16844.244695] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[16844.252128] <EOI> [<ffffffffa121cfd2>] ? osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[16844.261949] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[16844.271062] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16844.279969] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16844.288585] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16844.296993] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16844.304007] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16844.313097] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16844.319270] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16844.327300] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16844.334056] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16869.430125] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16869.439545] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16869.569471] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16869.582905] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16869.598980] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16869.608278] RIP: 0010:[<ffffffffa1162b46>] [<ffffffffa1162b46>] ldiskfs_itable_unused_count+0x16/0x30 [ldiskfs]
[16869.620619] RSP: 0018:ffff880703603d08 EFLAGS: 00000246
[16869.627485] RAX: 0000000000000000 RBX: 000000009f8f2001 RCX: ffff88042bd78800
[16869.636393] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[16869.645291] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[16869.654191] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16869.663097] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000246
[16869.671989] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16869.681950] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16869.689288] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16869.698188] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16869.707087] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16869.715980] Stack:
[16869.719155] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[16869.728400] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[16869.737665] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[16869.746903] Call Trace:
[16869.750596] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[16869.759738] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16869.768871] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16869.777710] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16869.786353] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16869.793529] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16869.802842] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16869.809233] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16869.817465] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16869.824440] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16869.832685] Code: 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 <48> 83 39 3f 76 07 0f b7 46 32 c1 e0 10 09 d0 5d c3 66 0f 1f 84
[16897.431101] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[16897.440751] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16897.572481] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16897.585765] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16897.602011] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16897.611475] RIP: 0010:[<ffffffffa121b9cd>] [<ffffffffa121b9cd>] osd_scrub_exec+0x3d/0x5b0 [osd_ldiskfs]
[16897.623131] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[16897.630060] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[16897.639035] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16897.647998] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[16897.656932] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16897.665871] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16897.674821] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16897.684841] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16897.692241] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16897.701212] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16897.710168] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16897.719111] Stack:
[16897.722323] ffff880703603d08 000000009f8f2001 ffff880703603df0 000000009f8f2001
[16897.731607] ffff880703603d57 000000009f8f2001 0000000000000004 0000000000000001
[16897.740914] 0000000000000004 ffff880703603d98 ffff880703603d78 0000000000000000
[16897.750234] Call Trace:
[16897.753952] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16897.763118] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16897.772283] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16897.781208] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16897.789904] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16897.797205] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16897.806550] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16897.812972] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16897.821250] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16897.828252] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16897.836513] Code: 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8d 43 08 48 85 db <4c> 0f 45 e0 41 83 f9 04 0f 84 84 00 00 00 41 83 f9 07 74 4f 41
[16925.432077] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16925.441759] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16925.573541] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16925.586831] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16925.603063] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16925.612478] RIP: 0010:[<ffffffffa1217b81>] [<ffffffffa1217b81>] osd_scrub_next+0x161/0x3c0 [osd_ldiskfs]
[16925.624225] RSP: 0018:ffff880703603ce8 EFLAGS: 00000282
[16925.631163] RAX: 0000000000000004 RBX: ffff880703603c78 RCX: ffff880703603d98
[16925.640138] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[16925.649095] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[16925.658036] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[16925.666984] R13: ffffffffa121ba79 R14: ffffffffffffff10 R15: ffff8803162f6000
[16925.675932] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16925.685951] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16925.693354] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16925.702308] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16925.711260] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16925.720199] Stack:
[16925.723397] ffff8800b2a3746c 0000000000000000 00000000ffffffff 0000000000000001
[16925.732699] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[16925.741984] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[16925.751263] Call Trace:
[16925.754968] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[16925.764149] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16925.773309] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16925.782229] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16925.790889] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16925.798097] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16925.807443] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16925.813870] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16925.822147] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16925.829156] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16925.837423] Code: 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b <41> 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0 0f
[16953.433051] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16953.442734] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16953.574440] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16953.587726] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16953.603988] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16953.613381] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[16953.625614] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[16953.632555] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[16953.641535] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16953.650504] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[16953.659459] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[16953.668408] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[16953.677357] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16953.687394] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16953.694806] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16953.703782] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16953.712740] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16953.721692] Stack:
[16953.724904] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[16953.734208] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[16953.743520] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[16953.752816] Call Trace:
[16953.756537] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16953.765699] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16953.774573] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16953.783249] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16953.790454] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16953.799803] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16953.806235] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16953.814515] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16953.821535] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16953.829814] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[16981.434027] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[16981.443714] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[16981.575611] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[16981.588919] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[16981.605215] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[16981.614627] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[16981.626290] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[16981.633248] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[16981.642235] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[16981.651212] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[16981.660173] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[16981.669134] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[16981.678084] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[16981.688110] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16981.695516] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[16981.704482] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16981.713450] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[16981.722405] Stack:
[16981.725612] ffff880703603df0 000000009f8f2001 ffffea00027e3c80 000000009f8f2001
[16981.734909] ffff880703603d57 ffff880703603d57 ffff8800b2a37468 0000000000000000
[16981.744192] ffff880703603d78 ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[16981.753487] Call Trace:
[16981.757209] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[16981.766378] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[16981.775535] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[16981.784401] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[16981.793063] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[16981.800265] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[16981.809605] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[16981.816036] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16981.824312] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[16981.831316] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[16981.839584] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[16987.725936] Lustre: soaked-MDT0000: Client a2eaa722-5c21-8192-3a30-b2265306fd45 (at 192.168.1.120@o2ib100) reconnecting
[16987.739262] Lustre: soaked-MDT0000: Connection restored to 91d08a06-6f67-40c2-ceb3-b8c39685af23 (at 192.168.1.120@o2ib100)
[17009.435002] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17009.444709] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17009.576378] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17009.589659] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17009.605932] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17009.615313] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[17009.626979] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[17009.633936] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[17009.642921] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17009.651910] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[17009.660886] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[17009.669867] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[17009.678856] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17009.688913] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17009.696341] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17009.705326] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17009.714350] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17009.723315] Stack:
[17009.726563] ffff880703603df0 0000000000000001 ffff880703603df0 000000009f8f2001
[17009.735867] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[17009.745189] 0000000000000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[17009.754508] Call Trace:
[17009.758259] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17009.767453] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17009.776637] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17009.785531] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17009.794242] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17009.801483] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17009.810864] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17009.817332] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17009.825686] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17009.832727] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17009.841032] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[17024.097520] INFO: rcu_sched self-detected stall on CPU { 24} (t=9600180 jiffies g=907112 c=907111 q=33376948)
[17024.110023] Task dump for CPU 24:
[17024.114766] OI_scrub R running task 0 5551 2 0x00000088
[17024.123690] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[17024.133012] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[17024.142313] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[17024.151591] Call Trace:
[17024.155270] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[17024.163242] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[17024.170197] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[17024.177714] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[17024.185401] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[17024.193650] [<ffffffff81099697>] update_process_times+0x47/0x80
[17024.201202] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[17024.209214] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[17024.216336] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[17024.223916] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[17024.231190] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[17024.238063] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[17024.246085] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[17024.253944] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[17024.261370] <EOI> [<ffffffffa121cfd2>] ? osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[17024.271184] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[17024.280293] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17024.289207] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17024.297822] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17024.306250] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17024.313203] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17024.322291] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17024.328460] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17024.336484] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17024.343262] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17049.436396] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17049.445822] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17049.575710] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17049.588929] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17049.605050] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17049.614418] RIP: 0010:[<ffffffffa1217a2b>] [<ffffffffa1217a2b>] osd_scrub_next+0xb/0x3c0 [osd_ldiskfs]
[17049.625921] RSP: 0018:ffff880703603d00 EFLAGS: 00000287
[17049.632790] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[17049.641698] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17049.650614] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17049.659512] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17049.668414] R13: 0000000000000018 R14: ffff880703603c78 R15: 0000000000000286
[17049.677305] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17049.687268] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17049.694607] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17049.703507] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17049.712409] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17049.721383] Stack:
[17049.724619] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[17049.733877] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[17049.743126] ffff8800b2a37468 0000000020000000 010000000000000c 0000000000000000
[17049.752391] Call Trace:
[17049.756082] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17049.765216] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17049.774409] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17049.783260] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17049.791902] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17049.799143] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17049.808554] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17049.815021] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17049.823257] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17049.830313] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17049.838550] Code: 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 <49> 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83
[17077.437371] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17077.447002] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17077.578721] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17077.592000] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17077.608249] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17077.617625] RIP: 0010:[<ffffffffa121cffc>] [<ffffffffa121cffc>] osd_inode_iteration+0x2ac/0xcc0 [osd_ldiskfs]
[17077.629850] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[17077.636777] RAX: 0000000000000004 RBX: ffff880703603d98 RCX: ffff880703603d98
[17077.645742] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[17077.654705] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[17077.663653] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[17077.672595] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[17077.681537] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17077.691555] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17077.698950] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17077.707908] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17077.716859] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17077.725810] Stack:
[17077.729007] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17077.738301] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17077.747599] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17077.756890] Call Trace:
[17077.760604] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17077.769765] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17077.778630] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17077.787311] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17077.794510] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17077.803854] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17077.810291] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17077.818572] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17077.825579] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17077.833848] Code: 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 <41> 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 0f
[17105.438348] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17105.448015] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17105.580201] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17105.593505] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17105.609793] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17105.619195] RIP: 0010:[<ffffffffa1217b80>] [<ffffffffa1217b80>] osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[17105.630956] RSP: 0018:ffff880703603ce0 EFLAGS: 00000282
[17105.637914] RAX: 0000000000000004 RBX: ffff880703603c78 RCX: ffff880703603d98
[17105.646900] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[17105.655876] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17105.664836] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[17105.673779] R13: ffffffffa121ba87 R14: ffffffffffffff10 R15: ffff8803162f6000
[17105.682720] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17105.692735] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17105.700139] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17105.709102] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17105.718062] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17105.727011] Stack:
[17105.730225] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[17105.739515] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[17105.748811] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[17105.758170] Call Trace:
[17105.761914] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17105.771082] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17105.780252] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17105.789133] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17105.797805] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17105.805023] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17105.814371] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17105.820807] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17105.829091] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17105.836100] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17105.844375] Code: 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0
[17133.439329] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17133.448999] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17133.580799] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17133.594078] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17133.610335] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17133.619718] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[17133.631392] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[17133.638333] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[17133.647307] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17133.656272] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[17133.665222] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[17133.674170] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[17133.683117] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17133.693154] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17133.700565] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17133.709533] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17133.718493] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17133.727446] Stack:
[17133.730665] ffff880703603df0 000000009f8f2001 ffff880703603df0 000000009f8f2001
[17133.739974] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[17133.749278] 0000000020000000 0000000000000000 ffff880703603d78 ffff8800b2a36000
[17133.758577] Call Trace:
[17133.762302] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17133.771465] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17133.780621] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17133.789486] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17133.798150] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17133.805361] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17133.814705] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17133.821144] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17133.829429] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17133.836436] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17133.844699] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[17154.584325] Lustre: soaked-MDT0000: Client af5c0dbc-bc84-dcde-3672-645b97702b86 (at 192.168.1.122@o2ib100) reconnecting
[17154.597698] Lustre: Skipped 1 previous similar message
[17154.604530] Lustre: soaked-MDT0000: Connection restored to 050e5cd6-4fff-0145-e167-2fb74af973b4 (at 192.168.1.122@o2ib100)
[17154.617921] Lustre: Skipped 1 previous similar message
[17161.440299] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17161.449988] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17161.581538] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17161.594805] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17161.611042] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17161.620425] RIP: 0010:[<ffffffffa1162b36>] [<ffffffffa1162b36>] ldiskfs_itable_unused_count+0x6/0x30 [ldiskfs]
[17161.632769] RSP: 0018:ffff880703603d08 EFLAGS: 00000286
[17161.639733] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[17161.648740] RDX: ffff880703603d78 RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[17161.657741] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17161.666746] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17161.675751] R13: ffff880703603d08 R14: ffff880703603c78 R15: 0000000000000296
[17161.684747] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17161.694816] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17161.702265] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17161.711274] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17161.720287] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17161.729284] Stack:
[17161.732544] ffff880703603df0 ffffffffa121cfb9 ffffffffa121b990 ffffffffa1217a20
[17161.741889] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[17161.751223] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[17161.760563] Call Trace:
[17161.764337] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[17161.773752] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17161.782970] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17161.791902] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17161.800628] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17161.807895] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17161.817302] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17161.823791] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17161.832133] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17161.839202] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17161.847520] Code: 00 00 31 c0 0f b7 56 10 48 89 e5 48 83 39 3f 76 07 0f b7 46 30 c1 e0 10 09 d0 5d c3 66 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 <48> 8b 8f a8 03 00 00 31 c0 0f b7 56 1c 48 89 e5 48 83 39 3f 76
[17189.441279] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17189.450964] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17189.582949] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17189.596242] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17189.612514] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17189.621903] RIP: 0010:[<ffffffffa121d203>] [<ffffffffa121d203>] osd_inode_iteration+0x4b3/0xcc0 [osd_ldiskfs]
[17189.634156] RSP: 0018:ffff880703603d18 EFLAGS: 00000216
[17189.641102] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[17189.650096] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17189.659075] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17189.668039] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[17189.676989] R13: ffff880703603df0 R14: 000000009f8f2001 R15: 0000000000000018
[17189.685938] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17189.695972] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17189.703383] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17189.712359] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17189.721333] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17189.730286] Stack:
[17189.733504] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17189.742786] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17189.752085] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17189.761382] Call Trace:
[17189.765113] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17189.774286] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17189.783158] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17189.791828] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17189.799035] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17189.808388] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17189.814827] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17189.823104] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17189.830105] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17189.838374] Code: ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff 41 3b 04 24 72 10 48 8b 85 50 ff ff ff <44> 3b 30 0f 87 95 fd ff ff 45 31 ff e9 a2 fc ff ff 0f 1f 40 00
[17204.106783] INFO: rcu_sched self-detected stall on CPU { 24} (t=9780183 jiffies g=907112 c=907111 q=34923589)
[17204.119225] Task dump for CPU 24:
[17204.123964] OI_scrub R running task 0 5551 2 0x00000088
[17204.132904] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[17204.142220] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[17204.151532] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[17204.160828] Call Trace:
[17204.164536] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[17204.172530] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[17204.179492] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[17204.187024] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[17204.194724] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[17204.202997] [<ffffffff81099697>] update_process_times+0x47/0x80
[17204.210561] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[17204.218603] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[17204.225739] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[17204.233351] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[17204.240639] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[17204.247520] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[17204.255551] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[17204.263381] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[17204.270810] <EOI> [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[17204.280616] [<ffffffffa121cfb9>] ? osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[17204.289708] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17204.298607] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17204.307224] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17204.315638] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17204.322589] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17204.331675] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17204.337853] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17204.345869] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17204.352649] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17229.442666] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17229.452086] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17229.581807] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17229.594980] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17229.611057] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17229.620361] RIP: 0010:[<ffffffffa121cfa5>] [<ffffffffa121cfa5>] osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[17229.632518] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[17229.639382] RAX: ffff8800b2a37468 RBX: 0000000000000000 RCX: 0000000000000000
[17229.648289] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17229.657187] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17229.666088] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[17229.674991] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[17229.683900] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17229.693869] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17229.701224] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17229.710135] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17229.719044] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17229.727940] Stack:
[17229.731112] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17229.740358] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17229.749614] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17229.758877] Call Trace:
[17229.762553] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17229.771682] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17229.780526] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17229.789171] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17229.796368] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17229.805696] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17229.812102] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17229.820362] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17229.827350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17229.835589] Code: 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 <48> 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42
[17257.443641] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17257.453249] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17257.585260] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17257.598565] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17257.614871] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17257.624286] RIP: 0010:[<ffffffffa1217b80>] [<ffffffffa1217b80>] osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[17257.636067] RSP: 0018:ffff880703603ce0 EFLAGS: 00000282
[17257.643032] RAX: 0000000000000004 RBX: ffff880703603c78 RCX: ffff880703603d98
[17257.652031] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[17257.661021] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17257.669986] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[17257.678931] R13: ffffffffa121ba64 R14: ffffffffffffff10 R15: ffff8803162f6000
[17257.687875] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17257.697902] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17257.705304] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17257.714269] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17257.723226] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17257.732176] Stack:
[17257.735386] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[17257.744660] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[17257.753962] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[17257.763253] Call Trace:
[17257.766980] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17257.776147] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17257.785309] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17257.794180] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17257.802863] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17257.810076] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17257.819428] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17257.825855] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17257.834138] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17257.841155] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17257.849428] Code: 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0
[17285.444617] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17285.454290] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17285.586174] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17285.599467] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17285.615727] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17285.625129] RIP: 0010:[<ffffffffa1217a35>] [<ffffffffa1217a35>] osd_scrub_next+0x15/0x3c0 [osd_ldiskfs]
[17285.636786] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[17285.643725] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[17285.652700] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17285.661663] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17285.670620] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[17285.679569] R13: ffff880703603d08 R14: 0000000000000010 R15: ffffffffa121ba87
[17285.688525] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17285.698557] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17285.705975] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17285.714945] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17285.723911] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17285.732859] Stack:
[17285.736081] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[17285.745386] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[17285.754702] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[17285.764021] Call Trace:
[17285.767748] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17285.776911] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17285.786081] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17285.794954] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17285.803621] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17285.810842] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17285.820201] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17285.826636] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17285.834937] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17285.841957] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17285.850238] Code: ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd <41> 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00
[17313.445599] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17313.455326] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17313.587069] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17313.600360] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17313.616614] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17313.626001] RIP: 0010:[<ffffffffa121b9c0>] [<ffffffffa121b9c0>] osd_scrub_exec+0x30/0x5b0 [osd_ldiskfs]
[17313.637656] RSP: 0018:ffff880703603c78 EFLAGS: 00000296
[17313.644599] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: 0000000000000000
[17313.653579] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17313.662552] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[17313.671508] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[17313.680459] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[17313.689418] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17313.699458] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17313.706858] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17313.715825] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17313.724782] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17313.733726] Stack:
[17313.736936] ffff880703603d08 0000000000000001 ffffea00027e3c80 000000009f8f2001
[17313.746207] ffff880703603d57 ffff880703603d57 ffffffffa121b990 0000000000000000
[17313.755511] ffff880703603d78 ffff8800b2a36000 ffff8803162f6000 ffffffffffffff10
[17313.764803] Call Trace:
[17313.768525] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17313.777691] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17313.786845] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17313.795989] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17313.804842] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17313.813503] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17313.820703] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17313.830040] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17313.836458] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17313.844728] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17313.851722] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17313.859980] Code: 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90 65 48 8b 04 25 28 00 00 00 <48> 89 45 d0 31 c0 48 8d 43 08 48 85 db 4c 0f 45 e0 41 83 f9 04
[17341.446567] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17341.456253] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17341.587740] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17341.600997] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17341.617187] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17341.626545] RIP: 0010:[<ffffffffa121d00e>] [<ffffffffa121d00e>] osd_inode_iteration+0x2be/0xcc0 [osd_ldiskfs]
[17341.638749] RSP: 0018:ffff880703603d18 EFLAGS: 00000297
[17341.645671] RAX: 0000000000000004 RBX: 0000000000000001 RCX: ffff880703603d98
[17341.654629] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[17341.663568] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[17341.672512] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17341.681448] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[17341.690395] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17341.700416] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17341.707811] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17341.716748] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17341.725693] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17341.734629] Stack:
[17341.737828] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17341.747131] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17341.756415] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17341.765690] Call Trace:
[17341.769396] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17341.778552] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17341.787400] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17341.796058] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17341.803260] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17341.812615] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17341.819042] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17341.827320] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17341.834328] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17341.842602] Code: 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 <66> 90 0f 84 8a 02 00 00 83 f8 06 0f 85 a1 01 00 00 48 8b 7d 90
[17369.447545] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17369.457240] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17369.589161] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17369.602474] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17369.618763] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17369.628167] RIP: 0010:[<ffffffffa121b995>] [<ffffffffa121b995>] osd_scrub_exec+0x5/0x5b0 [osd_ldiskfs]
[17369.639725] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[17369.646683] RAX: ffffffffa121b990 RBX: 0000000000000001 RCX: 0000000000000000
[17369.655666] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17369.664640] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17369.673598] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17369.682544] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[17369.691489] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17369.701512] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17369.708926] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17369.717891] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17369.726852] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17369.735797] Stack:
[17369.739007] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[17369.748290] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[17369.757577] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[17369.766867] Call Trace:
[17369.770589] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17369.779964] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17369.789128] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17369.798005] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17369.806679] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17369.813895] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17369.823256] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17369.829688] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17369.837968] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17369.844979] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17369.853247] Code: 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 <55> 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45 31 e4 53 48 83
[17384.116055] INFO: rcu_sched self-detected stall on CPU { 24} (t=9960186 jiffies g=907112 c=907111 q=36007586)
[17384.128510] Task dump for CPU 24:
[17384.133240] OI_scrub R running task 0 5551 2 0x00000088
[17384.142170] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[17384.151510] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[17384.160813] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[17384.170100] Call Trace:
[17384.173791] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[17384.181765] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[17384.188728] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[17384.196249] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[17384.203938] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[17384.212200] [<ffffffff81099697>] update_process_times+0x47/0x80
[17384.219758] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[17384.227774] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[17384.234899] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[17384.242491] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[17384.249759] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[17384.256644] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[17384.264672] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[17384.272501] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[17384.279927] <EOI> [<ffffffffa121cfa5>] ? osd_inode_iteration+0x255/0xcc0 [osd_ldiskfs]
[17384.289758] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17384.298859] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17384.307771] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17384.316389] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17384.324803] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17384.331757] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17384.340846] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17384.347022] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17384.355058] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17384.361818] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17409.448937] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17409.458413] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17409.588105] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17409.601286] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17409.617354] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17409.626652] RIP: 0010:[<ffffffffa121ba64>] [<ffffffffa121ba64>] osd_scrub_exec+0xd4/0x5b0 [osd_ldiskfs]
[17409.638225] RSP: 0018:ffff880703603c78 EFLAGS: 00000286
[17409.645086] RAX: ffff8803162f1008 RBX: ffff880703603c37 RCX: 0000000000000000
[17409.653998] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17409.662903] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[17409.671801] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[17409.680702] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[17409.689601] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17409.699577] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17409.706929] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17409.715842] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17409.724750] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17409.733649] Stack:
[17409.736818] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 ffff880703603d57
[17409.746072] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[17409.755323] ffff88042bd78800 ffff88042bd78800 00000000000072cf ffff880419dc3a00
[17409.764563] Call Trace:
[17409.768244] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17409.777376] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17409.786502] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17409.795337] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17409.803987] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17409.811171] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17409.820487] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17409.826869] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17409.835107] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17409.842079] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17409.850321] Code: 4d 88 4c 89 ef e8 1d fb 46 e0 41 83 87 68 14 00 00 01 49 83 87 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 <f6> 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00
[17437.449913] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17437.459488] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17437.591103] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17437.604392] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17437.620630] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17437.630012] RIP: 0010:[<ffffffffa1217a20>] [<ffffffffa1217a20>] osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17437.641760] RSP: 0018:ffff880703603d10 EFLAGS: 00000287
[17437.648700] RAX: ffffffffa1217a20 RBX: ffff880703603d98 RCX: ffff880703603d98
[17437.657669] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17437.666626] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[17437.675572] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[17437.684510] R13: 0000000000000004 R14: 000000009f8f2001 R15: 0000000000000018
[17437.693448] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17437.703452] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17437.710843] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17437.719800] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17437.728751] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17437.737678] Stack:
[17437.740870] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[17437.750142] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[17437.759436] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[17437.768723] Call Trace:
[17437.772419] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17437.781771] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17437.790926] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17437.799792] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17437.808453] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17437.815670] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17437.825015] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17437.831443] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17437.839717] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17437.846728] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17437.854994] Code: f7 48 8d 04 80 48 c1 e0 03 48 8d 4c 02 10 48 01 c2 e8 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd
[17465.450888] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17465.460565] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17465.592238] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17465.605522] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17465.621773] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17465.631157] RIP: 0010:[<ffffffffa121ba25>] [<ffffffffa121ba25>] osd_scrub_exec+0x95/0x5b0 [osd_ldiskfs]
[17465.642800] RSP: 0018:ffff880703603d08 EFLAGS: 00000292
[17465.649736] RAX: 0000000000000000 RBX: ffff880703603d57 RCX: 0000000000000000
[17465.658707] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17465.667668] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[17465.676619] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17465.685568] R13: ffffea00027e3c80 R14: ffff880703603c78 R15: ffff880703603d08
[17465.694519] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17465.704552] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17465.711957] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17465.720924] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17465.729884] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17465.738832] Stack:
[17465.742046] ffff880703603df0 ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20
[17465.751348] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[17465.760658] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[17465.769963] Call Trace:
[17465.773694] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17465.782872] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17465.792038] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17465.800911] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17465.809586] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17465.816791] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17465.826136] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17465.832572] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17465.840857] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17465.847857] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17465.856125] Code: 01 00 00 41 80 a7 74 14 00 00 fe 48 8b 4d d0 65 48 33 0c 25 28 00 00 00 0f 85 b1 03 00 00 48 83 c4 68 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 66 0f 1f 84 00 00 00 00 00 4c 8d ae f0 10 00 00 44 89 4d
[17493.451864] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17493.461515] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17493.593257] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17493.606540] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17493.622784] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17493.632165] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[17493.643811] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[17493.650757] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[17493.659730] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17493.668691] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[17493.677640] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[17493.686587] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[17493.695534] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17493.705562] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17493.712967] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17493.721932] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17493.730885] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17493.739827] Stack:
[17493.743035] ffff880703603d08 0000000000000001 ffffea00027e3c80 000000009f8f2001
[17493.752332] ffff880703603d57 ffff880703603d57 00000000000072ce ffff88042bd78800
[17493.761636] 00000000000072ce ffff880419dc3a00 ffff88042bd79800 ffffffffffffff10
[17493.770939] Call Trace:
[17493.774665] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17493.783830] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17493.792990] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17493.801875] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17493.810551] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17493.817765] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17493.827121] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17493.833559] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17493.841842] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17493.848852] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17493.857125] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[17521.452838] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17521.462554] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17521.594259] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17521.607548] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17521.623805] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17521.633190] RIP: 0010:[<ffffffffa1217b80>] [<ffffffffa1217b80>] osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[17521.644933] RSP: 0018:ffff880703603ce0 EFLAGS: 00000282
[17521.651874] RAX: 0000000000000004 RBX: ffff880703603c70 RCX: ffff880703603d98
[17521.660854] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[17521.669819] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17521.678769] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[17521.687714] R13: ffffffffa1217b6d R14: ffffffffffffff10 R15: ffff8803162f6000
[17521.696669] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17521.706705] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17521.714110] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17521.723074] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17521.732030] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17521.740974] Stack:
[17521.744214] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[17521.753517] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[17521.762820] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[17521.772127] Call Trace:
[17521.775847] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17521.785015] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17521.794178] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17521.803043] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17521.811720] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17521.818931] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17521.828288] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17521.834717] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17521.842993] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17521.850002] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17521.858271] Code: 10 00 00 02 00 00 00 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 <5b> 41 5c 41 5d 41 5e 41 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0
[17549.453813] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17549.463444] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17549.595297] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17549.608586] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17549.624848] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17549.634232] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[17549.645879] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[17549.652819] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[17549.661797] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17549.670760] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[17549.679713] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[17549.688660] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[17549.697615] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17549.707637] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17549.715038] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17549.724006] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17549.732967] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17549.741912] Stack:
[17549.745162] 0000000000000004 0000000000000001 0000000000000004 ffff880703603d98
[17549.754443] ffff880703603d57 0000000000000000 ffff8803162f6000 ffffffffffffff10
[17549.763734] ffffffffa1217b80 0000000000000010 0000000000000282 ffff880703603ce0
[17549.773025] Call Trace:
[17549.776745] [<ffffffffa1217b80>] ? osd_scrub_next+0x160/0x3c0 [osd_ldiskfs]
[17549.785618] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17549.794778] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17549.803931] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17549.812787] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17549.821451] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17549.828658] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17549.838009] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17549.844436] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17549.852710] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17549.859723] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17549.867991] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[17564.125324] INFO: rcu_sched self-detected stall on CPU { 24} (t=10140189 jiffies g=907112 c=907111 q=37172155)
[17564.137885] Task dump for CPU 24:
[17564.142603] OI_scrub R running task 0 5551 2 0x00000088
[17564.151530] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[17564.160831] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[17564.170119] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[17564.179390] Call Trace:
[17564.183065] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[17564.191026] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[17564.197963] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[17564.205457] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[17564.213128] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[17564.221357] [<ffffffff81099697>] update_process_times+0x47/0x80
[17564.228897] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[17564.236891] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[17564.243989] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[17564.251552] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[17564.258822] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[17564.265682] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[17564.273701] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[17564.281515] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[17564.288943] <EOI> [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17564.298273] [<ffffffffa1217a32>] ? osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[17564.306786] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17564.315689] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17564.324580] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17564.333187] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17564.341580] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17564.348522] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17564.357613] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17564.363780] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17564.371785] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17564.378527] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17589.455207] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17589.464618] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17589.594209] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17589.607379] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17589.623404] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17589.632678] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[17589.644822] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[17589.651677] RAX: ffff88042bd78800 RBX: 0000000000000001 RCX: ffff88042bd78800
[17589.660571] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[17589.669465] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17589.678361] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17589.687251] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[17589.696140] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17589.706102] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17589.713445] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17589.722355] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17589.731267] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17589.740163] Stack:
[17589.743331] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17589.752585] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17589.761840] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17589.771096] Call Trace:
[17589.774783] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17589.783919] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17589.792766] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17589.801424] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17589.808610] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17589.817942] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17589.824343] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17589.832598] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17589.839588] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17589.847836] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[17617.456183] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17617.465790] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17617.597696] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17617.611013] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17617.627317] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17617.636736] RIP: 0010:[<ffffffffa1217a32>] [<ffffffffa1217a32>] osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[17617.648425] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[17617.655392] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[17617.664392] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17617.673382] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17617.682345] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[17617.691305] R13: 0000000000000282 R14: 0000000000000010 R15: ffffffffa121ba79
[17617.700266] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17617.710295] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17617.717711] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17617.726676] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17617.735637] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17617.744582] Stack:
[17617.747799] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[17617.757077] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[17617.766353] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[17617.775641] Call Trace:
[17617.779364] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17617.788535] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17617.797700] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17617.806582] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17617.815274] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17617.822486] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17617.831839] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17617.838270] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17617.846558] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17617.853568] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17617.861844] Code: 25 fc ff ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <49> 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25
[17645.457159] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17645.466823] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17645.598490] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17645.611778] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17645.628032] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17645.637423] RIP: 0010:[<ffffffffa121ba79>] [<ffffffffa121ba79>] osd_scrub_exec+0xe9/0x5b0 [osd_ldiskfs]
[17645.649071] RSP: 0018:ffff880703603c78 EFLAGS: 00000282
[17645.656010] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[17645.664987] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17645.673953] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[17645.682913] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[17645.691863] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[17645.700815] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17645.710845] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17645.718250] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17645.727218] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17645.736176] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17645.745122] Stack:
[17645.748336] ffffea00027e3c80 000000009f8f2001 0000000000000004 0000000000000001
[17645.757611] ffff880703603d57 ffff880703603d98 ffff880703603d78 ffff8800b2a36000
[17645.766899] ffff8803162f6000 ffffffffffffff10 ffffffffa1217a32 0000000000000010
[17645.776195] Call Trace:
[17645.779931] [<ffffffffa1217a32>] ? osd_scrub_next+0x12/0x3c0 [osd_ldiskfs]
[17645.788714] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17645.797880] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17645.807037] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17645.815898] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17645.824569] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17645.831780] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17645.841129] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17645.847561] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17645.855839] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17645.862852] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17645.871131] Code: 88 11 00 00 01 4c 89 ef e8 f5 9f e9 df 44 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 <41> 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 41 f6 87 74 14
[17673.458133] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17673.467822] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17673.599502] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17673.612774] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17673.629009] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17673.638378] RIP: 0010:[<ffffffffa1217a35>] [<ffffffffa1217a35>] osd_scrub_next+0x15/0x3c0 [osd_ldiskfs]
[17673.650017] RSP: 0018:ffff880703603cf0 EFLAGS: 00000287
[17673.656954] RAX: ffffffffa1217a20 RBX: 000000009f8f2001 RCX: ffff880703603d98
[17673.665919] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17673.674876] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17673.683829] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[17673.692790] R13: 0000000000000286 R14: 0000000000000010 R15: ffffffffa121ba64
[17673.701743] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17673.711770] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17673.719177] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17673.728151] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17673.737101] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17673.746049] Stack:
[17673.749259] 0000000000000000 00000000ffffffff 0000000000000001 ffff880703603df0
[17673.758566] ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[17673.767851] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[17673.777142] Call Trace:
[17673.780869] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17673.790033] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17673.799187] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17673.808053] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17673.816722] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17673.823929] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17673.833280] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17673.839708] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17673.847990] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17673.855000] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17673.863267] Code: ff 48 83 c4 08 5b 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd <41> 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00
[17701.459109] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17701.468808] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17701.600530] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17701.613817] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17701.630071] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17701.639455] RIP: 0010:[<ffffffffa121d1f6>] [<ffffffffa121d1f6>] osd_inode_iteration+0x4a6/0xcc0 [osd_ldiskfs]
[17701.651687] RSP: 0018:ffff880703603d18 EFLAGS: 00000246
[17701.658629] RAX: 0000000020000000 RBX: 000000009f8f2001 RCX: 0000000000000000
[17701.667607] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17701.676575] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17701.685537] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[17701.694485] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffffea00027e3c80
[17701.703436] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17701.713465] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17701.720871] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17701.729838] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17701.738797] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17701.747743] Stack:
[17701.750956] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17701.760245] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17701.769552] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17701.778847] Call Trace:
[17701.782571] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17701.791737] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17701.800600] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17701.809274] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17701.816482] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17701.825831] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17701.832265] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17701.840537] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17701.847552] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17701.855830] Code: 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff ff <41> 3b 04 24 72 10 48 8b 85 50 ff ff ff 44 3b 30 0f 87 95 fd ff
[17724.700936] Lustre: 4594:0:(client.c:2114:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1494371069/real 1494371069] req@ffff8804d4400900 x1566943532435152/t0(0) o104->soaked-MDT0000@192.168.1.120@o2ib100:15/16 lens 296/224 e 0 to 1 dl 1494371076 ref 1 fl Rpc:X/0/ffffffff rc 0/-1
[17729.460085] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17729.469802] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17729.601731] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17729.615039] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17729.631338] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17729.640742] RIP: 0010:[<ffffffffa121cff0>] [<ffffffffa121cff0>] osd_inode_iteration+0x2a0/0xcc0 [osd_ldiskfs]
[17729.653020] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[17729.659998] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff880703603d98
[17729.669010] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17729.678018] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[17729.687022] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[17729.696029] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[17729.705037] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17729.715120] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17729.722574] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17729.731580] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17729.740583] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17729.749579] Stack:
[17729.752852] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17729.762211] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17729.771544] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17729.780879] Call Trace:
[17729.784648] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17729.793859] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17729.802775] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17729.811495] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17729.818755] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17729.828153] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17729.834633] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17729.842958] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17729.850025] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17729.858347] Code: 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff <48> 8b 85 30 ff ff ff ff d0 83 f8 03 41 89 c5 0f 84 4b 02 00 00
[17744.134595] INFO: rcu_sched self-detected stall on CPU { 24} (t=10320192 jiffies g=907112 c=907111 q=38226674)
[17744.147187] Task dump for CPU 24:
[17744.151935] OI_scrub R running task 0 5551 2 0x00000088
[17744.160874] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[17744.170209] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[17744.179508] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[17744.188823] Call Trace:
[17744.192538] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[17744.200530] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[17744.207501] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[17744.215041] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[17744.222761] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[17744.231029] [<ffffffff81099697>] update_process_times+0x47/0x80
[17744.238603] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[17744.246634] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[17744.253769] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[17744.261368] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[17744.268661] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[17744.275540] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[17744.283575] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[17744.291402] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[17744.298835] <EOI> [<ffffffffa121d1ee>] ? osd_inode_iteration+0x49e/0xcc0 [osd_ldiskfs]
[17744.308655] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17744.317743] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17744.326650] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17744.335267] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17744.343684] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17744.350639] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17744.359736] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17744.365913] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17744.373938] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17744.380699] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17769.461478] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17769.470909] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17769.600770] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17769.613945] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17769.630034] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17769.639376] RIP: 0010:[<ffffffffa1217a4f>] [<ffffffffa1217a4f>] osd_scrub_next+0x2f/0x3c0 [osd_ldiskfs]
[17769.650959] RSP: 0018:ffff880703603c70 EFLAGS: 00000282
[17769.657820] RAX: 00000000fb6db880 RBX: ffff880703603c37 RCX: ffff880703603d98
[17769.666738] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17769.675636] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17769.684536] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[17769.693437] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[17769.702332] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17769.712318] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17769.719672] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17769.728608] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17769.737518] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17769.746418] Stack:
[17769.749586] 000000009f8f2001 ffff880703603d08 000000009f8f2001 ffff880703603df0
[17769.758846] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[17769.768098] 0000000000000001 0000000000000004 ffff880703603d98 ffff880703603d78
[17769.777331] Call Trace:
[17769.781017] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17769.790151] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17769.799290] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17769.808205] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17769.816850] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17769.824031] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17769.833347] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17769.839748] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17769.848000] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17769.855059] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17769.863354] Code: 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 <31> c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b 41 f6 87 58 10 00 00
[17797.462453] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17797.472097] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17797.603810] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17797.617091] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17797.633327] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17797.642696] RIP: 0010:[<ffffffffa121cffc>] [<ffffffffa121cffc>] osd_inode_iteration+0x2ac/0xcc0 [osd_ldiskfs]
[17797.654919] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[17797.661846] RAX: 0000000000000004 RBX: 0000000000000001 RCX: ffff880703603d98
[17797.670817] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[17797.679783] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[17797.688727] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17797.697665] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[17797.706604] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17797.716624] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17797.724020] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17797.732964] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17797.741906] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17797.750835] Stack:
[17797.754031] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17797.763319] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17797.772648] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17797.781921] Call Trace:
[17797.785633] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17797.794842] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17797.803699] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17797.812364] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17797.819566] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17797.828924] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17797.835349] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17797.843623] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17797.850633] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17797.858899] Code: 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 <41> 89 c5 0f 84 4b 02 00 00 0f 8e 15 01 00 00 83 f8 05 66 90 0f
[17825.463429] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17825.473162] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17825.605007] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17825.618309] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17825.634598] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17825.643996] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[17825.656244] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[17825.663210] RAX: ffff8800b2a37468 RBX: 000000009f8f2001 RCX: 0000000000000000
[17825.672196] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17825.681171] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17825.690130] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17825.699078] R13: ffff880703603d08 R14: 000000009f8f2001 R15: 0000000000000018
[17825.708023] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17825.718053] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17825.725460] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17825.734434] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17825.743396] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17825.752345] Stack:
[17825.755560] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17825.764843] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17825.774134] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17825.783419] Call Trace:
[17825.787152] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17825.796319] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17825.805193] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17825.813875] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17825.821091] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17825.830453] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17825.836884] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17825.845163] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17825.852181] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17825.860455] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
[17853.464404] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[17853.474113] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17853.606034] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17853.619341] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17853.635633] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17853.645039] RIP: 0010:[<ffffffffa121cfda>] [<ffffffffa121cfda>] osd_inode_iteration+0x28a/0xcc0 [osd_ldiskfs]
[17853.657288] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[17853.664251] RAX: ffff88042bd78800 RBX: ffff880703603d98 RCX: ffff88042bd78800
[17853.673242] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[17853.682228] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[17853.691195] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[17853.700152] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[17853.709109] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17853.719133] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17853.726537] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17853.735503] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17853.744465] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17853.753418] Stack:
[17853.756629] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17853.765927] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17853.775194] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17853.784505] Call Trace:
[17853.788227] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17853.797390] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17853.806262] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17853.814946] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17853.822160] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17853.831516] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17853.837964] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17853.846241] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17853.853254] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17853.861526] Code: a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff <48> 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff
[17881.465386] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17881.475110] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17881.606878] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17881.620185] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17881.636499] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17881.645906] RIP: 0010:[<ffffffffa1217b89>] [<ffffffffa1217b89>] osd_scrub_next+0x169/0x3c0 [osd_ldiskfs]
[17881.657677] RSP: 0018:ffff880703603d08 EFLAGS: 00000282
[17881.664638] RAX: 0000000000000004 RBX: 0000000000000001 RCX: ffff880703603d98
[17881.673623] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[17881.682596] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[17881.691545] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17881.700490] R13: 0000000000000018 R14: ffff880703603c70 R15: 0000000000000282
[17881.709430] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17881.719443] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17881.726831] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17881.735786] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17881.744734] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17881.753668] Stack:
[17881.756867] ffff880703603df0 ffffffffa121cff9 ffffffffa121b990 ffffffffa1217a20
[17881.766143] ffff8800b2a36000 00000000810d354f ffff8803162f6000 ffff8800b2a37468
[17881.775422] 0000000020000000 010000000000000c 0000000000000000 0000000000000000
[17881.784697] Call Trace:
[17881.788415] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17881.797571] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17881.806720] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17881.815587] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17881.824254] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17881.831454] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17881.840789] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17881.847205] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17881.855472] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17881.862474] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17881.870737] Code: df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41 5c 41 5d 41 5e 41 5f <5d> c3 48 8b 05 be 58 9c ff 48 85 c0 0f 84 c2 fe ff ff 0f 1f 44
[17909.466356] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17909.475997] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17909.607665] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17909.620951] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17909.637183] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17909.646554] RIP: 0010:[<ffffffffa121d1e0>] [<ffffffffa121d1e0>] osd_inode_iteration+0x490/0xcc0 [osd_ldiskfs]
[17909.658773] RSP: 0018:ffff880703603d18 EFLAGS: 00000293
[17909.665710] RAX: 0000000000000004 RBX: 0000000000000001 RCX: 0000000000000000
[17909.674668] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17909.683684] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17909.692629] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17909.701569] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[17909.710514] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17909.720539] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17909.727944] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17909.736910] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17909.745872] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17909.754827] Stack:
[17909.758043] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17909.767333] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17909.776635] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17909.785931] Call Trace:
[17909.789652] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17909.798826] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17909.807701] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17909.816377] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17909.823587] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17909.832942] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17909.839377] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17909.847661] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17909.854693] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17909.862969] Code: 0f fc ff ff 0f 1f 80 00 00 00 00 45 89 e9 4c 8d 85 67 ff ff ff 48 8b 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff 48 8b bd 48 ff ff ff <48> 8b 85 28 ff ff ff ff d0 85 c0 41 89 c5 75 1c 8b 85 58 ff ff
[17924.143866] INFO: rcu_sched self-detected stall on CPU { 24} (t=10500195 jiffies g=907112 c=907111 q=39976003)
[17924.156427] Task dump for CPU 24:
[17924.161200] OI_scrub R running task 0 5551 2 0x00000088
[17924.170141] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[17924.179479] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[17924.188798] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[17924.198112] Call Trace:
[17924.201809] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[17924.209816] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[17924.216792] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[17924.224339] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[17924.232049] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[17924.240325] [<ffffffff81099697>] update_process_times+0x47/0x80
[17924.247898] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[17924.255944] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[17924.263085] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[17924.270688] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[17924.277984] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[17924.284874] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[17924.292919] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[17924.300765] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[17924.308197] <EOI> [<ffffffffa121cffc>] ? osd_inode_iteration+0x2ac/0xcc0 [osd_ldiskfs]
[17924.318022] [<ffffffffa121cff9>] ? osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[17924.327122] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17924.336029] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17924.344652] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17924.353067] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17924.360023] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17924.369112] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17924.375280] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17924.383306] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17924.390076] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17942.540728] Lustre: soaked-MDT0000: Client 6f6036ce-2e3a-7e95-1c07-11e5d5a627fe (at 192.168.1.124@o2ib100) reconnecting
[17942.554759] Lustre: soaked-MDT0000: Connection restored to 5a0e6555-4c2d-39c9-90c4-a332f62c8353 (at 192.168.1.124@o2ib100)
[17949.467749] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17949.477263] Modules linked in: osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17949.607241] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17949.620437] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17949.636528] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17949.645840] RIP: 0010:[<ffffffffa121cfd2>] [<ffffffffa121cfd2>] osd_inode_iteration+0x282/0xcc0 [osd_ldiskfs]
[17949.658010] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[17949.664901] RAX: ffff88042bd78800 RBX: ffff880703603d57 RCX: ffff88042bd78800
[17949.673842] RDX: 00000000000072cf RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[17949.682786] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17949.691722] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[17949.700651] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: 0000000000000018
[17949.709571] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17949.719575] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17949.726962] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17949.735907] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17949.744851] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17949.753787] Stack:
[17949.757005] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[17949.766304] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[17949.775604] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[17949.784906] Call Trace:
[17949.788632] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17949.797805] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17949.806697] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17949.815376] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17949.822592] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17949.831947] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17949.838383] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17949.846675] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17949.853692] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17949.861977] Code: 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff 42 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 <44> 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff
[17961.148283] FS-Cache: Loaded
[17961.189726] FS-Cache: Netfs 'nfs' registered for caching
[17961.207315] Key type dns_resolver registered
[17961.254191] NFS: Registering the id_resolver key type
[17961.261565] Key type id_resolver registered
[17961.268153] Key type id_legacy registered
[17977.468724] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[17977.478311] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[17977.613590] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[17977.626780] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[17977.642864] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[17977.652172] RIP: 0010:[<ffffffffa121b990>] [<ffffffffa121b990>] osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17977.664149] RSP: 0018:ffff880703603d10 EFLAGS: 00000293
[17977.671050] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[17977.679996] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[17977.688950] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[17977.697884] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000001
[17977.706821] R13: ffff880703603df0 R14: 000000009f8f2001 R15: 0000000000000018
[17977.715762] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[17977.725765] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[17977.733161] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[17977.742114] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[17977.751057] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[17977.759987] Stack:
[17977.763186] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[17977.772456] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[17977.781708] 010000000000000c 0000000000000000 0000000000000000 ffff8800b2a36000
[17977.790970] Call Trace:
[17977.794669] [<ffffffffa121d1e9>] ? osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[17977.804009] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[17977.813166] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[17977.822019] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[17977.830672] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[17977.837872] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[17977.847209] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[17977.853628] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17977.861883] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[17977.868862] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[17977.877095] Code: b5 07 00 00 c7 05 85 a5 02 00 00 00 02 00 89 54 24 08 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff <66> 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 41 54 45
[18005.469700] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[18005.479308] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18005.615655] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18005.628948] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18005.645206] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18005.654593] RIP: 0010:[<ffffffffa121b9a2>] [<ffffffffa121b9a2>] osd_scrub_exec+0x12/0x5b0 [osd_ldiskfs]
[18005.666251] RSP: 0018:ffff880703603cf0 EFLAGS: 00000293
[18005.673192] RAX: ffffffffa121b990 RBX: 000000009f8f2001 RCX: 0000000000000000
[18005.682171] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[18005.691154] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[18005.700113] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603c78
[18005.709067] R13: 0000000000000296 R14: 0000000000000010 R15: ffffffffa121b9ac
[18005.718026] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18005.728058] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18005.735463] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18005.744453] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18005.753410] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18005.762353] Stack:
[18005.765571] 0000000000000004 00000000ffffffff 0000000000000001 ffff880703603df0
[18005.774866] ffffffffa121d1e9 ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000
[18005.784174] 00000000810d354f ffff8803162f6000 ffff8800b2a37468 0000000020000000
[18005.793477] Call Trace:
[18005.797203] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[18005.806383] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18005.815558] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18005.824428] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18005.833095] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18005.840303] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18005.849649] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18005.856088] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18005.864374] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18005.871387] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18005.879662] Code: 48 8b 40 40 ba 07 00 00 00 48 89 04 24 31 c0 e8 a5 03 98 ff e9 c7 fd ff ff 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 <41> 54 45 31 e4 53 48 83 ec 68 48 8b 9e 08 10 00 00 4c 89 45 90
[18033.470682] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[18033.480387] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18033.616770] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18033.630066] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18033.646339] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18033.655725] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[18033.667392] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[18033.674332] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[18033.683308] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[18033.692273] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[18033.701226] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[18033.710182] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[18033.719132] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18033.729160] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18033.736580] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18033.745547] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18033.754516] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18033.763462] Stack:
[18033.766672] ffff880703603df0 000000009f8f2001 ffff880703603df0 000000009f8f2001
[18033.775976] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[18033.785280] ffffffffa121b990 0000000000000000 ffff880703603d78 ffff8800b2a36000
[18033.794573] Call Trace:
[18033.798291] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18033.807444] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[18033.816600] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18033.825752] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18033.834602] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18033.843269] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18033.850468] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18033.859809] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18033.866241] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18033.874507] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18033.881507] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18033.889779] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[18061.471650] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[18061.481372] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18061.617488] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18061.630748] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18061.646944] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18061.656301] RIP: 0010:[<ffffffffa1217a3b>] [<ffffffffa1217a3b>] osd_scrub_next+0x1b/0x3c0 [osd_ldiskfs]
[18061.667933] RSP: 0018:ffff880703603ce0 EFLAGS: 00000287
[18061.674862] RAX: ffffffffa1217a20 RBX: ffff880703603c78 RCX: ffff880703603d98
[18061.683817] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[18061.692753] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[18061.701692] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 0000000000000010
[18061.710627] R13: ffffffffa121ba89 R14: ffffffffffffff10 R15: ffff8803162f6000
[18061.719570] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18061.729586] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18061.736983] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18061.745938] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18061.754868] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18061.763795] Stack:
[18061.767002] ffff880419dc3a00 ffff8800b2a3746c 0000000000000000 00000000ffffffff
[18061.776306] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[18061.785591] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[18061.794874] Call Trace:
[18061.798586] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[18061.807748] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18061.816903] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18061.825759] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18061.834429] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18061.841637] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18061.850985] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18061.857422] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18061.865700] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18061.872711] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18061.880979] Code: 41 5c 41 5d 41 5e 5d c3 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 57 49 89 f7 41 56 41 55 49 89 fd 41 54 49 89 d4 53 <48> 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31
[18089.472625] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [OI_scrub:5551]
[18089.482275] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18089.618545] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18089.631834] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18089.648087] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18089.657473] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[18089.669122] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[18089.676073] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[18089.685050] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[18089.694024] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[18089.702986] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[18089.711937] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[18089.720891] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18089.730922] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18089.738336] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18089.747305] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18089.756272] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18089.765217] Stack:
[18089.768442] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[18089.777745] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[18089.787055] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[18089.796342] Call Trace:
[18089.800061] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[18089.809225] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18089.818380] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18089.827241] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18089.835898] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18089.843102] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18089.852447] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18089.858878] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18089.867150] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18089.874155] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18089.882419] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
[18104.153135] INFO: rcu_sched self-detected stall on CPU { 24} (t=10680198 jiffies g=907112 c=907111 q=41641355)
[18104.165705] Task dump for CPU 24:
[18104.170436] OI_scrub R running task 0 5551 2 0x00000088
[18104.179354] ffff88083fde6dd0 00000000fb6db880 ffff88082da03db0 ffffffff810c47e8
[18104.188664] 0000000000000018 ffffffff81a1e7c0 ffff88082da03dc8 ffffffff810c8079
[18104.197946] 0000000000000009 ffff88082da03df8 ffffffff811378e0 ffff88082da101c0
[18104.207233] Call Trace:
[18104.210913] <IRQ> [<ffffffff810c47e8>] sched_show_task+0xa8/0x110
[18104.218884] [<ffffffff810c8079>] dump_cpu_task+0x39/0x70
[18104.225845] [<ffffffff811378e0>] rcu_dump_cpu_stacks+0x90/0xd0
[18104.233358] [<ffffffff8113b032>] rcu_check_callbacks+0x442/0x720
[18104.241049] [<ffffffff810f35c0>] ? tick_sched_handle.isra.13+0x60/0x60
[18104.249299] [<ffffffff81099697>] update_process_times+0x47/0x80
[18104.256844] [<ffffffff810f3585>] tick_sched_handle.isra.13+0x25/0x60
[18104.264854] [<ffffffff810f3601>] tick_sched_timer+0x41/0x70
[18104.271968] [<ffffffff810b4d72>] __hrtimer_run_queues+0xd2/0x260
[18104.279547] [<ffffffff810b5310>] hrtimer_interrupt+0xb0/0x1e0
[18104.286830] [<ffffffff81698e5c>] ? call_softirq+0x1c/0x30
[18104.293702] [<ffffffff81050fd7>] local_apic_timer_interrupt+0x37/0x60
[18104.301742] [<ffffffff81699acf>] smp_apic_timer_interrupt+0x3f/0x60
[18104.309567] [<ffffffff8169801d>] apic_timer_interrupt+0x6d/0x80
[18104.316984] <EOI> [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18104.326595] [<ffffffffa121b9ac>] ? osd_scrub_exec+0x1c/0x5b0 [osd_ldiskfs]
[18104.335108] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[18104.344016] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18104.352910] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18104.361507] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18104.369900] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18104.376846] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18104.385924] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18104.392097] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18104.400105] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18104.406854] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18121.929925] INFO: task mount.nfs:8923 blocked for more than 120 seconds.
[18121.938469] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[18121.948031] mount.nfs D ffff8803ec731c00 0 8923 8922 0x00000080
[18121.956707] ffff88016f4a39a0 0000000000000082 ffff880186b7bec0 ffff88016f4a3fd8
[18121.965824] ffff88016f4a3fd8 ffff88016f4a3fd8 ffff880186b7bec0 ffff88016f4a3ae0
[18121.974919] ffff88016f4a3ae8 7fffffffffffffff ffff880186b7bec0 ffff8803ec731c00
[18121.984000] Call Trace:
[18121.987521] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18121.993827] [<ffffffff81689e29>] schedule_timeout+0x239/0x2c0
[18122.001090] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18122.007963] [<ffffffff8168c7a6>] wait_for_completion+0x116/0x170
[18122.015530] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18122.022525] [<ffffffff81137380>] ? dyntick_save_progress_counter+0x30/0x30
[18122.031035] [<ffffffff81116b3f>] __stop_cpus+0x5f/0x90
[18122.037565] [<ffffffff81137380>] ? dyntick_save_progress_counter+0x30/0x30
[18122.046060] [<ffffffff81116f3a>] try_stop_cpus+0x3a/0x60
[18122.052779] [<ffffffff811389e8>] synchronize_sched_expedited+0xd8/0x1d0
[18122.060960] [<ffffffff81138b8e>] synchronize_rcu_expedited+0xe/0x10
[18122.068735] [<ffffffff811a2e53>] bdi_unregister+0x143/0x1b0
[18122.075741] [<ffffffffa156f659>] nfs_put_super+0x19/0x20 [nfs]
[18122.083016] [<ffffffff81200d22>] generic_shutdown_super+0x72/0xf0
[18122.090571] [<ffffffff812010f2>] kill_anon_super+0x12/0x20
[18122.097456] [<ffffffffa156fa5b>] nfs_kill_super+0x1b/0x30 [nfs]
[18122.104812] [<ffffffff812014a9>] deactivate_locked_super+0x49/0x60
[18122.112462] [<ffffffff81201aa6>] deactivate_super+0x46/0x60
[18122.119438] [<ffffffff8121eef5>] mntput_no_expire+0xc5/0x120
[18122.126494] [<ffffffff8121ef74>] mntput+0x24/0x40
[18122.132468] [<ffffffff8121f0a8>] namespace_unlock+0x118/0x130
[18122.139631] [<ffffffff812219bb>] put_mnt_ns+0x4b/0x60
[18122.146023] [<ffffffff81221cc8>] mount_subtree+0x58/0xb0
[18122.152718] [<ffffffffa15c34a5>] nfs_follow_remote_path+0x115/0x1b0 [nfsv4]
[18122.161242] [<ffffffffa15c3686>] nfs4_try_mount+0x56/0xc0 [nfsv4]
[18122.168831] [<ffffffffa15644b7>] ? get_nfs_version+0x27/0x90 [nfs]
[18122.176490] [<ffffffffa15706c5>] nfs_fs_mount+0x4c5/0xd90 [nfs]
[18122.183862] [<ffffffffa15715e0>] ? nfs_clone_super+0x140/0x140 [nfs]
[18122.191704] [<ffffffffa156f5a0>] ? param_set_portnr+0x70/0x70 [nfs]
[18122.199441] [<ffffffff81202949>] mount_fs+0x39/0x1b0
[18122.205721] [<ffffffff811a5385>] ? __alloc_percpu+0x15/0x20
[18122.212671] [<ffffffff8121e6df>] vfs_kern_mount+0x5f/0xf0
[18122.219424] [<ffffffff81220c3e>] do_mount+0x24e/0xaa0
[18122.225791] [<ffffffff81221526>] SyS_mount+0x96/0xf0
[18122.232099] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18129.474019] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[18129.483336] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18129.616370] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18129.629409] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18129.645245] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18129.654442] RIP: 0010:[<ffffffffa121ba89>] [<ffffffffa121ba89>] osd_scrub_exec+0xf9/0x5b0 [osd_ldiskfs]
[18129.665934] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[18129.672732] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: 0000000000000000
[18129.681570] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[18129.690414] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[18129.699251] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[18129.708119] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[18129.716990] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18129.726925] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18129.734231] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18129.743112] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18129.751977] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18129.760852] Stack:
[18129.763973] 0000000000000004 0000000000000001 ffff880703603df0 ffff880703603d98
[18129.773170] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[18129.782376] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[18129.791583] Call Trace:
[18129.795254] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[18129.804347] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18129.813456] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18129.822276] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18129.830881] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18129.838027] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18129.847325] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18129.853709] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18129.861937] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18129.868895] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18129.877131] Code: 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 31 c0 <41> f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f 84 64
[18157.474993] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[18157.484611] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18157.621615] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18157.634911] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18157.651172] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18157.660649] RIP: 0010:[<ffffffffa121cfe9>] [<ffffffffa121cfe9>] osd_inode_iteration+0x299/0xcc0 [osd_ldiskfs]
[18157.672939] RSP: 0018:ffff880703603d18 EFLAGS: 00000287
[18157.679888] RAX: ffff88042bd78800 RBX: 000000009f8f2001 RCX: ffff880703603d98
[18157.688870] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff88042bd79800
[18157.697827] RBP: ffff880703603df0 R08: 0000000000000001 R09: 0000000000000004
[18157.706805] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[18157.715756] R13: ffff880703603d08 R14: 000000009f8f2001 R15: ffff880703603d08
[18157.724720] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18157.734751] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18157.742166] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18157.751149] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18157.760112] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18157.769056] Stack:
[18157.772267] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[18157.781558] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[18157.790859] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[18157.800156] Call Trace:
[18157.803896] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18157.813072] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18157.821947] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18157.830632] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18157.837842] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18157.847195] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18157.853633] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18157.861916] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18157.868934] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18157.877209] Code: 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83 57 01 00 00 44 0f b6 85 67 ff ff ff 48 8d 4d a8 48 8d 55 88 48 8b b5 38 ff ff ff <48> 8b bd 48 ff ff ff 48 8b 85 30 ff ff ff ff d0 83 f8 03 41 89
[18185.475976] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[18185.485699] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18185.622997] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18185.636311] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18185.652617] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18185.662030] RIP: 0010:[<ffffffffa121cfa1>] [<ffffffffa121cfa1>] osd_inode_iteration+0x251/0xcc0 [osd_ldiskfs]
[18185.674287] RSP: 0018:ffff880703603d18 EFLAGS: 00000286
[18185.681262] RAX: ffff8800b2a37468 RBX: 0000000000000000 RCX: 0000000000000000
[18185.690258] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[18185.699243] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[18185.708212] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffff880703603d57
[18185.717171] R13: 0000000000000004 R14: 000000009f8f2001 R15: ffffea00027e3c80
[18185.726124] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18185.736151] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18185.743558] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18185.752525] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18185.761487] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18185.770433] Stack:
[18185.773643] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[18185.782931] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[18185.792223] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[18185.801523] Call Trace:
[18185.805255] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18185.814420] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18185.823290] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18185.831964] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18185.839181] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18185.848526] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18185.854954] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18185.863239] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18185.870248] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18185.878518] Code: b0 00 00 00 5b 41 5c 41 5d 41 5e 41 5f 5d c3 0f 1f 00 8b 75 98 48 8b 7d 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 <44> 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77
<ConMan> Console [soak-8] joined by <cliffwhi@ssh-1> on pts/4 at 05-09 23:12.
CentOS Linux 7 (Core)
Kernel 3.10.0-514.16.1.el7_lustre.x86_64 on an x86_64
soak-8 login: root
Password:
Last login: Tue May 9 23:07:26 from soak-35.spirit.hpdd.intel.com
[root@soak-8 ~]# echo t > /proc/syst[Krq-trigger [18213.476945] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
[18213.486566] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm; ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18213.623160] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18213.636504] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18213.652800] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18213.662216] RIP: 0010:[<ffffffffa121ba87>] [<ffffffffa121ba87>] osd_scrub_exec+0xf7/0x5b0 [osd_ldiskfs]
[18213.673899] RSP: 0018:ffff880703603c78 EFLAGS: 00000246
[18213.680859] RAX: 0000000000280001 RBX: ffff880703603c37 RCX: 0000000000000000
[18213.689849] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[18213.698840] RBP: ffff880703603d08 R08: ffff880703603d57 R09: 0000000000000004
[18213.707812] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[18213.716763] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[18213.725713] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18213.735749] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18213.743156] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18213.752132] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18213.761671] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18213.770617] Stack:
[18213.773815] ffffea00027e3c80 000000009f8f2001 ffff880703603df0 ffff880703603d57
[18213.783107] ffff880703603d57 000000009f8f2001 0000000000000004 ffff880703603d57
[18213.792389] ffff8800b2a37468 0000000000000000 ffff880703603d78 ffff8800b2a36000
[18213.801688] Call Trace:
[18213.805422] [<ffffffffa121d1e9>] osd_inode_iteration+0x499/0xcc0 [osd_ldiskfs]
[18213.814585] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18213.823752] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18213.832624] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18213.841286] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18213.848484] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18213.857824] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18213.864245] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18213.872513] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18213.879504] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18213.887765] Code: 8b 4d 88 48 85 db 74 23 f6 83 18 0a 00 00 08 74 1a 4d 85 e4 74 15 41 8b 87 6c 14 00 00 41 39 84 24 0c 0a 00 00 0f 82 d9 00 00 00 <31> c0 41 f6 87 74 14 00 00 04 0f 85 6e ff ff ff 41 83 f9 02 0f
Message from syslogd@soak-8 at May 9 23:12:45 ...
kernel:NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
;echo c > /proc/sysrq-trigger
-bash: syntax error near unexpected token `;;'
[root@soak-8 ~]# echo t > /proc/sysrq-trigger ;;echo c > /proc/sysrq-trigger [1P
[18236.263970] SysRq : Show State
[18236.268673] task PC stack pid father
[18236.276321] systemd S 0000000000000000 0 1 0 0x00000000
[18236.285228] ffff88017a1a3db8 0000000000000082 ffff8804ba508000 ffff88017a1a3fd8
[18236.294541] ffff88017a1a3fd8 ffff88017a1a3fd8 ffff8804ba508000 0000000000000000
[18236.303836] 0000000000000000 ffff88082cb85260 ffff8804ba508000 0000000000000000
[18236.313113] Call Trace:
[18236.316781] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18236.323241] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18236.332031] [<ffffffff812484a9>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[18236.340499] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18236.348939] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18236.355860] [<ffffffff8124adfd>] ? do_timerfd_settime+0x2ed/0x3a0
[18236.363635] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18236.370644] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18236.377630] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18236.385087] kthreadd S ffff8804ba508fb0 0 2 0 0x00000000
[18236.393724] ffff88017a1a7e70 0000000000000046 ffff8804ba508fb0 ffff88017a1a7fd8
[18236.402768] ffff88017a1a7fd8 ffff88017a1a7fd8 ffff8804ba508fb0 ffffffff819fdd40
[18236.411793] ffff88061efabd38 ffff8804ba508fb0 ffff8804ba508fb0 ffff8804ba508fb0
[18236.420811] Call Trace:
[18236.424240] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18236.430571] [<ffffffff810b1586>] kthreadd+0x2e6/0x2f0
[18236.437019] [<ffffffff810b12a0>] ? kthread_create_on_cpu+0x60/0x60
[18236.444802] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18236.451525] [<ffffffff810b12a0>] ? kthread_create_on_cpu+0x60/0x60
[18236.459213] ksoftirqd/0 S 0000000000000000 0 3 2 0x00000000
[18236.467812] ffff88017a1b7e80 0000000000000046 ffff88017a248000 ffff88017a1b7fd8
[18236.476825] ffff88017a1b7fd8 ffff88017a1b7fd8 ffff88017a248000 ffff88017a248000
[18236.485816] ffff88017fc7c260 ffffffff819f1100 0000000000000000 0000000000000000
[18236.494834] Call Trace:
[18236.498266] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18236.504495] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18236.511695] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18236.518897] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18236.525028] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.533003] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18236.539699] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.547660] kworker/0:0H S ffff88042e016840 0 5 2 0x00000000
[18236.556257] ffff88017a257e50 0000000000000046 ffff88017a249f60 ffff88017a257fd8
[18236.565247] ffff88017a257fd8 ffff88017a257fd8 ffff88017a249f60 ffff88042e016860
[18236.574234] ffff8804bfc40eb0 ffff88017a249f60 ffff8804bfc40e80 ffff88042e016840
[18236.583216] Call Trace:
[18236.586612] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18236.592829] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18236.599719] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18236.606908] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18236.612985] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.620909] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18236.627564] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.635479] migration/0 S 0000000000000000 0 8 2 0x00000000
[18236.644016] ffff88017a263e80 0000000000000046 ffff88017a24ce70 ffff88017a263fd8
[18236.652962] ffff88017a263fd8 ffff88017a263fd8 ffff88017a24ce70 ffff88017a24ce70
[18236.661907] ffff88017fc7c280 ffffffff81a08f40 0000000000000000 0000000000000000
[18236.670848] Call Trace:
[18236.674206] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18236.680384] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18236.687529] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18236.694673] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18236.700743] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.708656] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18236.715324] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.723262] rcu_bh S 0000000000000000 0 9 2 0x00000000
[18236.731818] ffff88017a267e10 0000000000000046 ffff88017a24de20 ffff88017a267fd8
[18236.740779] ffff88017a267fd8 ffff88017a267fd8 ffff88017a24de20 ffffffff81a09d80
[18236.749716] ffff88017a24de20 ffffffff81a1e3e0 0000000000000000 0000000000000000
[18236.758657] Call Trace:
[18236.762028] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18236.768218] [<ffffffff8113a4c2>] rcu_gp_kthread+0xb2/0x7e0
[18236.775081] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18236.782199] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18236.789639] [<ffffffff8113a410>] ? rcu_gp_fqs+0xb0/0xb0
[18236.796201] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18236.802287] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.810207] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18236.816860] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.824785] rcu_sched S 000000000000f888 0 10 2 0x00000000
[18236.833318] ffff88017a26bd60 0000000000000046 ffff88017a24edd0 ffff88017a26bfd8
[18236.842290] ffff88017a26bfd8 ffff88017a26bfd8 ffff88017a24edd0 ffff8801795fc000
[18236.851237] ffff88017a26bd90 000000010111ad8b ffff8801795fc000 000000000000f888
[18236.860190] Call Trace:
[18236.863549] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18236.869735] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18236.877280] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18236.884624] [<ffffffff810b1786>] ? prepare_to_wait+0x56/0x90
[18236.891718] [<ffffffff8113a759>] rcu_gp_kthread+0x349/0x7e0
[18236.898866] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18236.906018] [<ffffffff8113a410>] ? rcu_gp_fqs+0xb0/0xb0
[18236.912567] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18236.918629] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.926532] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18236.933172] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18236.941080] watchdog/0 S 0000000000000000 0 11 2 0x00000000
[18236.949603] ffff88017a26fe80 0000000000000046 ffff88017a270000 ffff88017a26ffd8
[18236.958544] ffff88017a26ffd8 ffff88017a26ffd8 ffff88017a270000 ffff88017a270000
[18236.967472] ffff88017fc7c780 ffffffff81a09820 0000000000000000 0000000000000000
[18236.976430] Call Trace:
[18236.979796] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18236.985981] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18236.993112] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.000336] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.006413] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.014317] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.020954] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.028866] watchdog/1 S 0000000000000000 0 12 2 0x00000000
[18237.037388] ffff880179d2be80 0000000000000046 ffff880179d20000 ffff880179d2bfd8
[18237.046345] ffff880179d2bfd8 ffff880179d2bfd8 ffff880179d20000 ffff880179d20000
[18237.055304] ffff88017fc7c790 ffffffff81a09820 0000000000000000 0000000000000000
[18237.064264] Call Trace:
[18237.067624] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.073792] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.080923] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.088057] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.094114] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.102027] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.108651] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.116551] migration/1 S 0000000000000000 0 13 2 0x00000000
[18237.125077] ffff880179d2fe80 0000000000000046 ffff880179d20fb0 ffff880179d2ffd8
[18237.134035] ffff880179d2ffd8 ffff880179d2ffd8 ffff880179d20fb0 ffff880179d20fb0
[18237.142973] ffff88017fc7c7a0 ffffffff81a08f40 0000000000000000 0000000000000000
[18237.151927] Call Trace:
[18237.155292] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.161466] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.168611] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.175738] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.181800] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.189712] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.196368] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.204284] ksoftirqd/1 S 0000000000000000 0 14 2 0x00000000
[18237.212811] ffff880179d33e80 0000000000000046 ffff880179d21f60 ffff880179d33fd8
[18237.221760] ffff880179d33fd8 ffff880179d33fd8 ffff880179d21f60 ffff880179d21f60
[18237.231067] ffff88017fc7c7b0 ffffffff819f1100 0000000000000000 0000000000000000
[18237.240064] Call Trace:
[18237.243416] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.249576] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.256726] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.263882] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.269980] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.277926] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.284588] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.292510] kworker/1:0H S ffff88042e056840 0 16 2 0x00000000
[18237.301065] ffff880179d4be50 0000000000000046 ffff880179d23ec0 ffff880179d4bfd8
[18237.310024] ffff880179d4bfd8 ffff880179d4bfd8 ffff880179d23ec0 ffff88042e056860
[18237.318985] ffff8804bfc41630 ffff880179d23ec0 ffff8804bfc41600 ffff88042e056840
[18237.327939] Call Trace:
[18237.331300] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.337487] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18237.344347] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18237.351498] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.357575] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.365520] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.372210] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.380182] watchdog/2 S 0000000000000000 0 17 2 0x00000000
[18237.388713] ffff8801795dbe80 0000000000000046 ffff880179d24e70 ffff8801795dbfd8
[18237.397673] ffff8801795dbfd8 ffff8801795dbfd8 ffff880179d24e70 ffff880179d24e70
[18237.406618] ffff88017fc7c7c0 ffffffff81a09820 0000000000000000 0000000000000000
[18237.415575] Call Trace:
[18237.418962] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.425169] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.432334] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.439489] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.445571] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.453503] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.460176] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.468103] migration/2 S 0000000000000000 0 18 2 0x00000000
[18237.476636] ffff8801795dfe80 0000000000000046 ffff880179d25e20 ffff8801795dffd8
[18237.485592] ffff8801795dffd8 ffff8801795dffd8 ffff880179d25e20 ffff880179d25e20
[18237.494547] ffff88017fc7c7d0 ffffffff81a08f40 0000000000000000 0000000000000000
[18237.503501] Call Trace:
[18237.506863] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.513052] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.520217] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.527373] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.533459] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.541388] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.548062] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.555996] ksoftirqd/2 S 0000000000000000 0 19 2 0x00000000
[18237.564534] ffff8801795ebe80 0000000000000046 ffff880179d26dd0 ffff8801795ebfd8
[18237.573502] ffff8801795ebfd8 ffff8801795ebfd8 ffff880179d26dd0 ffff880179d26dd0
[18237.582459] ffff88017fc7c7e0 ffffffff819f1100 0000000000000000 0000000000000000
[18237.591411] Call Trace:
[18237.594767] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.600962] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.608129] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.615286] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.621372] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.629307] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.635984] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.643919] kworker/2:0H S ffff88042e096840 0 21 2 0x00000000
[18237.652456] ffff8801795fbe50 0000000000000046 ffff8801795f0fb0 ffff8801795fbfd8
[18237.661416] ffff8801795fbfd8 ffff8801795fbfd8 ffff8801795f0fb0 ffff88042e096860
[18237.670744] ffff8804b9daf3b0 ffff8801795f0fb0 ffff8804b9daf380 ffff88042e096840
[18237.679679] Call Trace:
[18237.683057] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.689258] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18237.696128] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18237.703288] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.709370] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.717308] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.723971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.731915] watchdog/3 S 0000000000000000 0 22 2 0x00000000
[18237.740453] ffff880179633e80 0000000000000046 ffff8801795f1f60 ffff880179633fd8
[18237.749400] ffff880179633fd8 ffff880179633fd8 ffff8801795f1f60 ffff8801795f1f60
[18237.758365] ffff88017fc7c7f0 ffffffff81a09820 0000000000000000 0000000000000000
[18237.767317] Call Trace:
[18237.770674] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.776862] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.784026] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.791194] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.797285] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.805221] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.811887] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.819814] migration/3 S 0000000000000000 0 23 2 0x00000000
[18237.828364] ffff880179637e80 0000000000000046 ffff8801795f2f10 ffff880179637fd8
[18237.837481] ffff880179637fd8 ffff880179637fd8 ffff8801795f2f10 ffff8801795f2f10
[18237.846440] ffff88017fc7c800 ffffffff81a08f40 0000000000000000 0000000000000000
[18237.855407] Call Trace:
[18237.858772] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.864964] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.872118] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.879289] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.885381] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.893318] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.899992] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.907916] ksoftirqd/3 S 0000000000000000 0 24 2 0x00000000
[18237.916455] ffff88017963be80 0000000000000046 ffff8801795f3ec0 ffff88017963bfd8
[18237.925424] ffff88017963bfd8 ffff88017963bfd8 ffff8801795f3ec0 ffff8801795f3ec0
[18237.934365] ffff88017fc7c810 ffffffff819f1100 0000000000000000 0000000000000000
[18237.943326] Call Trace:
[18237.946690] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18237.952878] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18237.960041] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18237.967193] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18237.973273] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.981207] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18237.987885] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18237.995807] kworker/3:0H S ffff88042e0d6840 0 26 2 0x00000000
[18238.004359] ffff880179643e50 0000000000000046 ffff8801795f5e20 ffff880179643fd8
[18238.013315] ffff880179643fd8 ffff880179643fd8 ffff8801795f5e20 ffff88042e0d6860
[18238.022280] ffff8804b9daf030 ffff8801795f5e20 ffff8804b9daf000 ffff88042e0d6840
[18238.031219] Call Trace:
[18238.034577] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.040758] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18238.047618] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18238.054770] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.060863] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.068798] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.075461] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.083386] watchdog/4 S 0000000000000000 0 27 2 0x00000000
[18238.092295] ffff88017965be80 0000000000000046 ffff8801795f6dd0 ffff88017965bfd8
[18238.101309] ffff88017965bfd8 ffff88017965bfd8 ffff8801795f6dd0 ffff8801795f6dd0
[18238.110277] ffff88017fc7c820 ffffffff81a09820 0000000000000000 0000000000000000
[18238.119222] Call Trace:
[18238.122587] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.128769] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18238.135940] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18238.143105] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.149197] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.157126] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.163802] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.171727] migration/4 S 0000000000000000 0 28 2 0x00000000
[18238.180280] ffff88017965fe80 0000000000000046 ffff880179660000 ffff88017965ffd8
[18238.189234] ffff88017965ffd8 ffff88017965ffd8 ffff880179660000 ffff880179660000
[18238.198194] ffff88017fc7c830 ffffffff81a08f40 0000000000000000 0000000000000000
[18238.207140] Call Trace:
[18238.210498] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.216684] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18238.223846] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18238.231000] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.237081] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.245126] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.251905] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.259847] ksoftirqd/4 S 0000000000000000 0 29 2 0x00000000
[18238.268399] ffff88017966be80 0000000000000046 ffff880179660fb0 ffff88017966bfd8
[18238.277373] ffff88017966bfd8 ffff88017966bfd8 ffff880179660fb0 ffff880179660fb0
[18238.286332] ffff88017fc7c840 ffffffff819f1100 0000000000000000 0000000000000000
[18238.295284] Call Trace:
[18238.298646] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.304834] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18238.311990] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18238.319157] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.325259] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.333198] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.339868] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.347799] kworker/4:0 S ffff88042e116480 0 30 2 0x00000000
[18238.356353] ffff88017966fe50 0000000000000046 ffff880179661f60 ffff88017966ffd8
[18238.365318] ffff88017966ffd8 ffff88017966ffd8 ffff880179661f60 ffff88042e1164a0
[18238.374288] ffff8804b9daedb0 ffff880179661f60 ffff8804b9daed80 ffff88042e116480
[18238.383231] Call Trace:
[18238.386586] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.392762] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18238.399621] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18238.406769] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.412857] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.420779] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.427448] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.435381] kworker/4:0H S ffff88042e116840 0 31 2 0x00000000
[18238.443931] ffff880179673e50 0000000000000046 ffff880179662f10 ffff880179673fd8
[18238.452881] ffff880179673fd8 ffff880179673fd8 ffff880179662f10 ffff88042e116860
[18238.461835] ffff8804b9daecb0 ffff880179662f10 ffff8804b9daec80 ffff88042e116840
[18238.470766] Call Trace:
[18238.474126] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.480317] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18238.487181] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18238.494328] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.500408] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.508330] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.514997] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.522921] watchdog/5 S 0000000000000000 0 32 2 0x00000000
[18238.531452] ffff88017968fe80 0000000000000046 ffff880179663ec0 ffff88017968ffd8
[18238.540780] ffff88017968ffd8 ffff88017968ffd8 ffff880179663ec0 ffff880179663ec0
[18238.549735] ffff88017fc7c850 ffffffff81a09820 0000000000000000 0000000000000000
[18238.558697] Call Trace:
[18238.562066] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.568262] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18238.575410] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18238.582557] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.588635] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.596558] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.603218] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.611139] migration/5 S 0000000000000000 0 33 2 0x00000000
[18238.619671] ffff880179693e80 0000000000000046 ffff880179664e70 ffff880179693fd8
[18238.628639] ffff880179693fd8 ffff880179693fd8 ffff880179664e70 ffff880179664e70
[18238.637578] ffff88017fc7c860 ffffffff81a08f40 0000000000000000 0000000000000000
[18238.646523] Call Trace:
[18238.649890] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.656078] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18238.663228] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18238.670385] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.676468] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.684408] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.691069] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.698989] ksoftirqd/5 S 0000000000000000 0 34 2 0x00000000
[18238.707517] ffff880179697e80 0000000000000046 ffff880179665e20 ffff880179697fd8
[18238.716464] ffff880179697fd8 ffff880179697fd8 ffff880179665e20 ffff880179665e20
[18238.725413] ffff88017fc7c870 ffffffff819f1100 0000000000000000 0000000000000000
[18238.734371] Call Trace:
[18238.737730] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.743931] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18238.751076] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18238.758229] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.764322] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.772257] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.778920] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.786843] kworker/5:0H S ffff88042e156840 0 36 2 0x00000000
[18238.795387] ffff8801796a7e50 0000000000000046 ffff8801796b0000 ffff8801796a7fd8
[18238.804356] ffff8801796a7fd8 ffff8801796a7fd8 ffff8801796b0000 ffff88042e156860
[18238.813313] ffff880179d5ae30 ffff8801796b0000 ffff880179d5ae00 ffff88042e156840
[18238.822268] Call Trace:
[18238.825624] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.831803] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18238.838659] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18238.845808] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.851896] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.859819] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.866480] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.874408] watchdog/6 S 0000000000000000 0 37 2 0x00000000
[18238.882963] ffff8801796bfe80 0000000000000046 ffff8801796b0fb0 ffff8801796bffd8
[18238.891920] ffff8801796bffd8 ffff8801796bffd8 ffff8801796b0fb0 ffff8801796b0fb0
[18238.900885] ffff88017fc7c880 ffffffff81a09820 0000000000000000 0000000000000000
[18238.909823] Call Trace:
[18238.913196] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18238.919383] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18238.926545] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18238.933692] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18238.939772] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.947702] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18238.954370] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18238.962657] migration/6 S 0000000000000000 0 38 2 0x00000000
[18238.971223] ffff8801796d3e80 0000000000000046 ffff8801796b1f60 ffff8801796d3fd8
[18238.980208] ffff8801796d3fd8 ffff8801796d3fd8 ffff8801796b1f60 ffff8801796b1f60
[18238.989161] ffff88017fc7c890 ffffffff81a08f40 0000000000000000 0000000000000000
[18238.998118] Call Trace:
[18239.001478] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.007666] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.014816] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.022087] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.028387] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.036349] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.043026] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.050980] ksoftirqd/6 S 0000000000000000 0 39 2 0x00000000
[18239.059528] ffff8801796d7e80 0000000000000046 ffff8801796b2f10 ffff8801796d7fd8
[18239.068501] ffff8801796d7fd8 ffff8801796d7fd8 ffff8801796b2f10 ffff8801796b2f10
[18239.077447] ffff88017fc7c8a0 ffffffff819f1100 0000000000000000 0000000000000000
[18239.086391] Call Trace:
[18239.089749] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.095931] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.103098] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.110251] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.116332] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.124263] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.130936] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.138875] kworker/6:0H S ffff88042e196840 0 41 2 0x00000000
[18239.147404] ffff8801796dfe50 0000000000000046 ffff8801796b4e70 ffff8801796dffd8
[18239.156373] ffff8801796dffd8 ffff8801796dffd8 ffff8801796b4e70 ffff88042e196860
[18239.165331] ffff880179d5b1b0 ffff8801796b4e70 ffff880179d5b180 ffff88042e196840
[18239.174290] Call Trace:
[18239.177646] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.183824] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18239.190679] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18239.197829] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.203918] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.211845] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.218506] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.226437] watchdog/7 S 0000000000000000 0 42 2 0x00000000
[18239.234979] ffff8801796fbe80 0000000000000046 ffff8801796b5e20 ffff8801796fbfd8
[18239.243947] ffff8801796fbfd8 ffff8801796fbfd8 ffff8801796b5e20 ffff8801796b5e20
[18239.252899] ffff88017fc7c8b0 ffffffff81a09820 0000000000000000 0000000000000000
[18239.261852] Call Trace:
[18239.265221] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.271404] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.278561] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.285731] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.291809] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.299735] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.306406] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.314328] migration/7 S 0000000000000000 0 43 2 0x00000000
[18239.322879] ffff8801796ffe80 0000000000000046 ffff8801796b6dd0 ffff8801796fffd8
[18239.331822] ffff8801796fffd8 ffff8801796fffd8 ffff8801796b6dd0 ffff8801796b6dd0
[18239.340768] ffff88017fc7c8c0 ffffffff81a08f40 0000000000000000 0000000000000000
[18239.349734] Call Trace:
[18239.353102] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.359299] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.366451] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.373605] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.379675] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.387611] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.394283] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.402219] ksoftirqd/7 S 0000000000000000 0 44 2 0x00000000
[18239.410751] ffff880179713e80 0000000000000046 ffff880179708000 ffff880179713fd8
[18239.419714] ffff880179713fd8 ffff880179713fd8 ffff880179708000 ffff880179708000
[18239.428653] ffff88017fc7c8d0 ffffffff819f1100 0000000000000000 0000000000000000
[18239.437942] Call Trace:
[18239.441658] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.447839] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.454995] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.462153] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.468252] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.476185] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.482844] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.490761] kworker/7:0H S ffff88042e1d6840 0 46 2 0x00000000
[18239.499509] ffff88017971be50 0000000000000046 ffff880179709f60 ffff88017971bfd8
[18239.508632] ffff88017971bfd8 ffff88017971bfd8 ffff880179709f60 ffff88042e1d6860
[18239.517592] ffff8801796e0330 ffff880179709f60 ffff8801796e0300 ffff88042e1d6840
[18239.526526] Call Trace:
[18239.529891] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.536072] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18239.542938] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18239.550092] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.556170] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.564097] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.570756] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.578675] watchdog/8 S 0000000000000000 0 47 2 0x00000000
[18239.587209] ffff8804b962be80 0000000000000046 ffff8804b9da0fb0 ffff8804b962bfd8
[18239.596166] ffff8804b962bfd8 ffff8804b962bfd8 ffff8804b9da0fb0 ffff8804b9da0fb0
[18239.605116] ffff8804ba74e000 ffffffff81a09820 0000000000000000 0000000000000000
[18239.614068] Call Trace:
[18239.617426] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.623604] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.630752] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.637917] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.644004] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.651930] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.658593] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.666513] migration/8 S 0000000000000000 0 48 2 0x00000000
[18239.675055] ffff8804b962fe80 0000000000000046 ffff8804b9da1f60 ffff8804b962ffd8
[18239.684004] ffff8804b962ffd8 ffff8804b962ffd8 ffff8804b9da1f60 ffff8804b9da1f60
[18239.692968] ffff8804ba74e010 ffffffff81a08f40 0000000000000000 0000000000000000
[18239.701911] Call Trace:
[18239.705285] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.711470] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.718619] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.725769] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.731845] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.739769] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.746434] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.754360] ksoftirqd/8 S 0000000000000000 0 49 2 0x00000000
[18239.762912] ffff8804b9633e80 0000000000000046 ffff8804b9da2f10 ffff8804b9633fd8
[18239.771853] ffff8804b9633fd8 ffff8804b9633fd8 ffff8804b9da2f10 ffff8804b9da2f10
[18239.780798] ffff8804ba74e020 ffffffff819f1100 0000000000000000 0000000000000000
[18239.789742] Call Trace:
[18239.793115] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.799303] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.806458] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.813609] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.819687] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.827613] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.834276] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.842211] kworker/8:0H S ffff88082d816840 0 51 2 0x00000000
[18239.850741] ffff8804b963be50 0000000000000046 ffff8804b9da4e70 ffff8804b963bfd8
[18239.859698] ffff8804b963bfd8 ffff8804b963bfd8 ffff8804b9da4e70 ffff88082d816860
[18239.868650] ffff8801796e06b0 ffff8804b9da4e70 ffff8801796e0680 ffff88082d816840
[18239.877597] Call Trace:
[18239.880967] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.887177] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18239.894056] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18239.901209] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.907298] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.915240] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18239.921916] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18239.929841] watchdog/9 S 0000000000000000 0 53 2 0x00000000
[18239.938379] ffff8804b9667e80 0000000000000046 ffff8804b9668000 ffff8804b9667fd8
[18239.947345] ffff8804b9667fd8 ffff8804b9667fd8 ffff8804b9668000 ffff8804b9668000
[18239.956339] ffff8804ba74e030 ffffffff81a09820 0000000000000000 0000000000000000
[18239.965344] Call Trace:
[18239.968703] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18239.974889] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18239.982042] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18239.989202] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18239.995288] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.003215] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.009883] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.017798] migration/9 S 0000000000000000 0 54 2 0x00000000
[18240.026330] ffff8804b9673e80 0000000000000046 ffff8804b9668fb0 ffff8804b9673fd8
[18240.035296] ffff8804b9673fd8 ffff8804b9673fd8 ffff8804b9668fb0 ffff8804b9668fb0
[18240.044252] ffff8804ba74e040 ffffffff81a08f40 0000000000000000 0000000000000000
[18240.053210] Call Trace:
[18240.056566] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.062747] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18240.069905] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18240.077056] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.083144] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.091084] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.097742] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.105660] ksoftirqd/9 S 0000000000000000 0 55 2 0x00000000
[18240.114192] ffff8804b9677e80 0000000000000046 ffff8804b9669f60 ffff8804b9677fd8
[18240.123158] ffff8804b9677fd8 ffff8804b9677fd8 ffff8804b9669f60 ffff8804b9669f60
[18240.132118] ffff8804ba74e050 ffffffff819f1100 0000000000000000 0000000000000000
[18240.141077] Call Trace:
[18240.144449] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.150626] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18240.157780] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18240.164941] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.171029] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.178963] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.185623] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.193541] kworker/9:0 S ffff88082d856480 0 56 2 0x00000000
[18240.202076] ffff8804b9683e50 0000000000000046 ffff8804b966af10 ffff8804b9683fd8
[18240.211038] ffff8804b9683fd8 ffff8804b9683fd8 ffff8804b966af10 ffff88082d8564a0
[18240.220013] ffff8801796e0a30 ffff8804b966af10 ffff8801796e0a00 ffff88082d856480
[18240.228956] Call Trace:
[18240.232320] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.238507] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18240.245377] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18240.252524] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.258599] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.266528] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.273188] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.281108] kworker/9:0H S ffff88082d856840 0 57 2 0x00000000
[18240.289650] ffff8804b9687e50 0000000000000046 ffff8804b966bec0 ffff8804b9687fd8
[18240.298602] ffff8804b9687fd8 ffff8804b9687fd8 ffff8804b966bec0 ffff88082d856860
[18240.307566] ffff8801796e0b30 ffff8804b966bec0 ffff8801796e0b00 ffff88082d856840
[18240.316505] Call Trace:
[18240.319866] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.326042] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18240.332897] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18240.340056] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.346141] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.354074] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.360729] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.368651] watchdog/10 S 0000000000000000 0 58 2 0x00000000
[18240.377184] ffff8804b968fe80 0000000000000046 ffff8804b966ce70 ffff8804b968ffd8
[18240.386143] ffff8804b968ffd8 ffff8804b968ffd8 ffff8804b966ce70 ffff8804b966ce70
[18240.395083] ffff8804ba74e060 ffffffff81a09820 0000000000000000 0000000000000000
[18240.404032] Call Trace:
[18240.407395] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.413569] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18240.420714] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18240.427866] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.433944] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.441865] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.448526] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.456449] migration/10 S 0000000000000000 0 59 2 0x00000000
[18240.465000] ffff8804b96a3e80 0000000000000046 ffff8804b966de20 ffff8804b96a3fd8
[18240.473941] ffff8804b96a3fd8 ffff8804b96a3fd8 ffff8804b966de20 ffff8804b966de20
[18240.482908] ffff8804ba74e070 ffffffff81a08f40 0000000000000000 0000000000000000
[18240.491870] Call Trace:
[18240.495238] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.501426] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18240.508574] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18240.515722] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.521803] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.529727] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.536387] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.544308] ksoftirqd/10 S 0000000000000000 0 60 2 0x00000000
[18240.552824] ffff8804b96a7e80 0000000000000046 ffff8804b966edd0 ffff8804b96a7fd8
[18240.561770] ffff8804b96a7fd8 ffff8804b96a7fd8 ffff8804b966edd0 ffff8804b966edd0
[18240.570723] ffff8804ba74e080 ffffffff819f1100 0000000000000000 0000000000000000
[18240.579664] Call Trace:
[18240.583043] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.589230] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18240.596382] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18240.603541] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.609620] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.617552] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.624223] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.632153] kworker/10:0 S ffff88082d896480 0 61 2 0x00000000
[18240.640692] ffff8804b96b3e50 0000000000000046 ffff8804b96a8000 ffff8804b96b3fd8
[18240.649643] ffff8804b96b3fd8 ffff8804b96b3fd8 ffff8804b96a8000 ffff88082d8964a0
[18240.658612] ffff8801796e0db0 ffff8804b96a8000 ffff8801796e0d80 ffff88082d896480
[18240.667561] Call Trace:
[18240.670929] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.677120] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18240.683987] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18240.691150] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.697233] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.705167] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.711827] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.719755] kworker/10:0H S ffff88082d896840 0 62 2 0x00000000
[18240.728305] ffff8804b96b7e50 0000000000000046 ffff8804b96a8fb0 ffff8804b96b7fd8
[18240.737279] ffff8804b96b7fd8 ffff8804b96b7fd8 ffff8804b96a8fb0 ffff88082d896860
[18240.746234] ffff8801796e0eb0 ffff8804b96a8fb0 ffff8801796e0e80 ffff88082d896840
[18240.755190] Call Trace:
[18240.758546] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.764722] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18240.771575] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18240.778722] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.784798] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.792717] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.799368] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.807293] watchdog/11 S 0000000000000000 0 63 2 0x00000000
[18240.815822] ffff8804b96d3e80 0000000000000046 ffff8804b96a9f60 ffff8804b96d3fd8
[18240.824775] ffff8804b96d3fd8 ffff8804b96d3fd8 ffff8804b96a9f60 ffff8804b96a9f60
[18240.833725] ffff8804ba74e090 ffffffff81a09820 0000000000000000 0000000000000000
[18240.842663] Call Trace:
[18240.846036] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.852219] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18240.859374] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18240.866520] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.872597] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.880525] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.887199] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.895130] migration/11 S 0000000000000000 0 64 2 0x00000000
[18240.903662] ffff8804b96d7e80 0000000000000046 ffff8804b96aaf10 ffff8804b96d7fd8
[18240.912614] ffff8804b96d7fd8 ffff8804b96d7fd8 ffff8804b96aaf10 ffff8804b96aaf10
[18240.921555] ffff8804ba74e0a0 ffffffff81a08f40 0000000000000000 0000000000000000
[18240.930514] Call Trace:
[18240.933868] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18240.940059] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18240.947213] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18240.954385] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18240.960471] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.968410] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18240.975075] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18240.983007] ksoftirqd/11 S 0000000000000000 0 65 2 0x00000000
[18240.991532] ffff8804b96dbe80 0000000000000046 ffff8804b96abec0 ffff8804b96dbfd8
[18241.000548] ffff8804b96dbfd8 ffff8804b96dbfd8 ffff8804b96abec0 ffff8804b96abec0
[18241.009588] ffff8804ba74e0b0 ffffffff819f1100 0000000000000000 0000000000000000
[18241.018544] Call Trace:
[18241.021908] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18241.028098] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18241.035252] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18241.042416] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18241.048505] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.056434] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18241.063105] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.071047] kworker/11:0 S ffff88082d8d6480 0 66 2 0x00000000
[18241.079590] ffff8804b96dfe50 0000000000000046 ffff8804b96ace70 ffff8804b96dffd8
[18241.088542] ffff8804b96dffd8 ffff8804b96dffd8 ffff8804b96ace70 ffff88082d8d64a0
[18241.097506] ffff8801796e1130 ffff8804b96ace70 ffff8801796e1100 ffff88082d8d6480
[18241.106450] Call Trace:
[18241.109808] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18241.116004] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18241.122861] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18241.130024] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18241.136120] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.144071] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18241.150739] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.158672] kworker/11:0H S ffff88082d8d6840 0 67 2 0x00000000
[18241.167219] ffff8804b96ebe50 0000000000000046 ffff8804b96ade20 ffff8804b96ebfd8
[18241.176184] ffff8804b96ebfd8 ffff8804b96ebfd8 ffff8804b96ade20 ffff88082d8d6860
[18241.185148] ffff8801796e1230 ffff8804b96ade20 ffff8801796e1200 ffff88082d8d6840
[18241.194112] Call Trace:
[18241.197531] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18241.203709] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18241.210565] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18241.217721] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18241.223809] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.231733] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18241.238414] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.246341] watchdog/12 S 0000000000000000 0 68 2 0x00000000
[18241.254873] ffff8804b9727e80 0000000000000046 ffff8804b96aedd0 ffff8804b9727fd8
[18241.263834] ffff8804b9727fd8 ffff8804b9727fd8 ffff8804b96aedd0 ffff8804b96aedd0
[18241.272798] ffff8804ba74e0c0 ffffffff81a09820 0000000000000000 0000000000000000
[18241.281745] Call Trace:
[18241.285120] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18241.291307] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18241.298476] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18241.305622] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18241.311703] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.319627] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18241.326294] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.334221] migration/12 S 0000000000000000 0 69 2 0x00000000
[18241.342752] ffff8804b9733e80 0000000000000046 ffff8804b9728000 ffff8804b9733fd8
[18241.351704] ffff8804b9733fd8 ffff8804b9733fd8 ffff8804b9728000 ffff8804b9728000
[18241.360650] ffff8804ba74e0d0 ffffffff81a08f40 0000000000000000 0000000000000000
[18241.369577] Call Trace:
[18241.372937] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18241.379121] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18241.386268] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18241.393423] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18241.399511] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.407443] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18241.414114] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.422047] ksoftirqd/12 S 0000000000000000 0 70 2 0x00000000
[18241.430569] ffff8804b9737e80 0000000000000046 ffff8804b9728fb0 ffff8804b9737fd8
[18241.439530] ffff8804b9737fd8 ffff8804b9737fd8 ffff8804b9728fb0 ffff8804b9728fb0
[18241.448470] ffff8804ba74e0e0 ffffffff819f1100 0000000000000000 0000000000000000
[18241.457430] Call Trace:
[18241.460790] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18241.466988] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18241.474141] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18241.477925] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
Message from[18241.477961] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support syslogd@soak-8 at May 9 23:13:13 ...
kernel mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
:NMI watchdog: BUG: soft lockup - CPU#24 stuck f[18241.477982] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
or 22s! [OI_scru[18241.477983] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
b:5551]
[18241.477983] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18241.478004] RIP: 0010:[<ffffffffa1217a5d>] [<ffffffffa1217a5d>] osd_scrub_next+0x3d/0x3c0 [osd_ldiskfs]
[18241.478005] RSP: 0018:ffff880703603c70 EFLAGS: 00000246
[18241.478006] RAX: 0000000000000000 RBX: ffff880703603c37 RCX: ffff880703603d98
[18241.478006] RDX: ffff880703603d78 RSI: ffff8800b2a36000 RDI: ffff8803162f6000
[18241.478007] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[18241.478008] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: ffffffffa12184d1
[18241.478008] R13: ffff880703603c68 R14: ffff8800b2a37110 R15: ffff88009f8f2dc0
[18241.478009] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18241.478010] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18241.478011] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18241.478011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18241.478012] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18241.478012] Stack:
[18241.478013] 000000009f8f2001 ffff880703603d08 ffff880703603d57 ffff880703603df0
[18241.478014] 000000009f8f2001 ffff880703603d57 000000009f8f2001 0000000000000004
[18241.478015] 0000000000000001 0000000000000004 ffff880703603d98 ffff880703603d78
[18241.478015] Call Trace:
[18241.478024] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[18241.478031] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18241.478037] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18241.478043] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18241.478047] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18241.478053] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18241.478055] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18241.478056] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.478058] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18241.478059] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.478069] Code: 49 89 fd 41 54 49 89 d4 53 48 89 cb 48 83 ec 70 65 48 8b 04 25 28 00 00 00 48 89 45 d0 31 c0 48 8b 05 f8 59 9c ff 48 85 c0 75 7b <41> f6 87 58 10 00 00 08 0f 84 62 03 00 00 4d 8d b7 e0 10 00 00
[18241.909868] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18241.916280] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.924543] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18241.931533] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18241.939792] kworker/12:0H S ffff88082d916840 0 72 2 0x00000000
[18241.948664] ffff8804b973fe50 0000000000000046 ffff8804b972af10 ffff8804b973ffd8
[18241.957906] ffff8804b973ffd8 ffff8804b973ffd8 ffff8804b972af10 ffff88082d916860
[18241.967134] ffff8801796e15b0 ffff8804b972af10 ffff8801796e1580 ffff88082d916840
[18241.976338] Call Trace:
[18241.979951] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18241.986346] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18241.993404] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18242.000738] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18242.006980] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.015039] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18242.021811] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.029830] watchdog/13 S 0000000000000000 0 73 2 0x00000000
[18242.038493] ffff8804b9747e80 0000000000000046 ffff8804b972bec0 ffff8804b9747fd8
[18242.047558] ffff8804b9747fd8 ffff8804b9747fd8 ffff8804b972bec0 ffff8804b972bec0
[18242.056585] ffff8804ba74e0f0 ffffffff81a09820 0000000000000000 0000000000000000
[18242.065589] Call Trace:
[18242.069003] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.075238] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18242.082459] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18242.089661] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18242.095789] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.103757] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18242.110456] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.118423] migration/13 S 0000000000000000 0 74 2 0x00000000
[18242.127005] ffff8804b975be80 0000000000000046 ffff8804b972ce70 ffff8804b975bfd8
[18242.136013] ffff8804b975bfd8 ffff8804b975bfd8 ffff8804b972ce70 ffff8804b972ce70
[18242.145018] ffff8804ba74e100 ffffffff81a08f40 0000000000000000 0000000000000000
[18242.154010] Call Trace:
[18242.157419] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.163643] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18242.170830] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18242.178022] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18242.184162] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.192123] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18242.198818] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.206778] ksoftirqd/13 S 0000000000000000 0 75 2 0x00000000
[18242.215359] ffff8804b975fe80 0000000000000046 ffff8804b972de20 ffff8804b975ffd8
[18242.224339] ffff8804b975ffd8 ffff8804b975ffd8 ffff8804b972de20 ffff8804b972de20
[18242.233339] ffff8804ba74e110 ffffffff819f1100 0000000000000000 0000000000000000
[18242.242325] Call Trace:
[18242.244110] INFO: task mount.nfs:8923 blocked for more than 120 seconds.
[18242.244111] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[18242.244113] mount.nfs D ffff8803ec731c00 0 8923 8922 0x00000080
[18242.244115] ffff88016f4a39a0 0000000000000082 ffff880186b7bec0 ffff88016f4a3fd8
[18242.244116] ffff88016f4a3fd8 ffff88016f4a3fd8 ffff880186b7bec0 ffff88016f4a3ae0
[18242.244118] ffff88016f4a3ae8 7fffffffffffffff ffff880186b7bec0 ffff8803ec731c00
[18242.244118] Call Trace:
[18242.244123] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.244125] [<ffffffff81689e29>] schedule_timeout+0x239/0x2c0
[18242.244127] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18242.244130] [<ffffffff8168c7a6>] wait_for_completion+0x116/0x170
[18242.244134] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18242.244138] [<ffffffff81137380>] ? dyntick_save_progress_counter+0x30/0x30
[18242.244140] [<ffffffff81116b3f>] __stop_cpus+0x5f/0x90
[18242.244142] [<ffffffff81137380>] ? dyntick_save_progress_counter+0x30/0x30
[18242.244143] [<ffffffff81116f3a>] try_stop_cpus+0x3a/0x60
[18242.244146] [<ffffffff811389e8>] synchronize_sched_expedited+0xd8/0x1d0
[18242.244147] [<ffffffff81138b8e>] synchronize_rcu_expedited+0xe/0x10
[18242.244152] [<ffffffff811a2e53>] bdi_unregister+0x143/0x1b0
[18242.244168] [<ffffffffa156f659>] nfs_put_super+0x19/0x20 [nfs]
[18242.244172] [<ffffffff81200d22>] generic_shutdown_super+0x72/0xf0
[18242.244174] [<ffffffff812010f2>] kill_anon_super+0x12/0x20
[18242.244179] [<ffffffffa156fa5b>] nfs_kill_super+0x1b/0x30 [nfs]
[18242.244180] [<ffffffff812014a9>] deactivate_locked_super+0x49/0x60
[18242.244182] [<ffffffff81201aa6>] deactivate_super+0x46/0x60
[18242.244185] [<ffffffff8121eef5>] mntput_no_expire+0xc5/0x120
[18242.244187] [<ffffffff8121ef74>] mntput+0x24/0x40
[18242.244189] [<ffffffff8121f0a8>] namespace_unlock+0x118/0x130
[18242.244191] [<ffffffff812219bb>] put_mnt_ns+0x4b/0x60
[18242.244193] [<ffffffff81221cc8>] mount_subtree+0x58/0xb0
[18242.244209] [<ffffffffa15c34a5>] nfs_follow_remote_path+0x115/0x1b0 [nfsv4]
[18242.244215] [<ffffffffa15c3686>] nfs4_try_mount+0x56/0xc0 [nfsv4]
[18242.244219] [<ffffffffa15644b7>] ? get_nfs_version+0x27/0x90 [nfs]
[18242.244225] [<ffffffffa15706c5>] nfs_fs_mount+0x4c5/0xd90 [nfs]
[18242.244230] [<ffffffffa15715e0>] ? nfs_clone_super+0x140/0x140 [nfs]
[18242.244234] [<ffffffffa156f5a0>] ? param_set_portnr+0x70/0x70 [nfs]
[18242.244236] [<ffffffff81202949>] mount_fs+0x39/0x1b0
[18242.244238] [<ffffffff811a5385>] ? __alloc_percpu+0x15/0x20
[18242.244240] [<ffffffff8121e6df>] vfs_kern_mount+0x5f/0xf0
[18242.244242] [<ffffffff81220c3e>] do_mount+0x24e/0xaa0
[18242.244244] [<ffffffff81221526>] SyS_mount+0x96/0xf0
[18242.244245] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18242.551724] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.557835] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18242.564915] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18242.572009] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18242.578044] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.585882] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18242.592455] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.600275] kworker/13:0H S ffff88082d956840 0 77 2 0x00000000
[18242.608709] ffff8804b9767e50 0000000000000046 ffff8804b9768000 ffff8804b9767fd8
[18242.617550] ffff8804b9767fd8 ffff8804b9767fd8 ffff8804b9768000 ffff88082d956860
[18242.626393] ffff8801796e1930 ffff8804b9768000 ffff8801796e1900 ffff88082d956840
[18242.635250] Call Trace:
[18242.638512] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.644595] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18242.651359] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18242.658426] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18242.664423] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.672275] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18242.678836] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.686669] watchdog/14 S 0000000000000000 0 78 2 0x00000000
[18242.695129] ffff8804b9793e80 0000000000000046 ffff8804b9768fb0 ffff8804b9793fd8
[18242.704009] ffff8804b9793fd8 ffff8804b9793fd8 ffff8804b9768fb0 ffff8804b9768fb0
[18242.712872] ffff8804ba74e120 ffffffff81a09820 0000000000000000 0000000000000000
[18242.721743] Call Trace:
[18242.725039] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.731154] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18242.738232] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18242.745326] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18242.751332] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.759198] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18242.765793] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.773657] migration/14 S 0000000000000000 0 79 2 0x00000000
[18242.782140] ffff8804b9797e80 0000000000000046 ffff8804b9769f60 ffff8804b9797fd8
[18242.791052] ffff8804b9797fd8 ffff8804b9797fd8 ffff8804b9769f60 ffff8804b9769f60
[18242.799953] ffff8804ba74e130 ffffffff81a08f40 0000000000000000 0000000000000000
[18242.808862] Call Trace:
[18242.812194] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.818346] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18242.825469] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18242.832579] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18242.838623] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.846513] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18242.853150] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.861053] ksoftirqd/14 S 0000000000000000 0 80 2 0x00000000
[18242.869570] ffff8804b979be80 0000000000000046 ffff8804b976af10 ffff8804b979bfd8
[18242.878524] ffff8804b979bfd8 ffff8804b979bfd8 ffff8804b976af10 ffff8804b976af10
[18242.887479] ffff8804ba74e140 ffffffff819f1100 0000000000000000 0000000000000000
[18242.896436] Call Trace:
[18242.899798] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.905983] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18242.913134] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18242.920285] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18242.926370] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.934313] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18242.940980] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18242.948897] kworker/14:0H S ffff88082d996840 0 82 2 0x00000000
[18242.957436] ffff8804b97a3e50 0000000000000046 ffff8804b976ce70 ffff8804b97a3fd8
[18242.966397] ffff8804b97a3fd8 ffff8804b97a3fd8 ffff8804b976ce70 ffff88082d996860
[18242.975361] ffff8801796e1cb0 ffff8804b976ce70 ffff8801796e1c80 ffff88082d996840
[18242.984314] Call Trace:
[18242.987676] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18242.993853] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18243.000708] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18243.007864] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.013943] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.021858] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.028538] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.036466] watchdog/15 S 0000000000000000 0 83 2 0x00000000
[18243.044995] ffff8804b97bbe80 0000000000000046 ffff8804b976de20 ffff8804b97bbfd8
[18243.054039] ffff8804b97bbfd8 ffff8804b97bbfd8 ffff8804b976de20 ffff8804b976de20
[18243.063032] ffff8804ba74e150 ffffffff81a09820 0000000000000000 0000000000000000
[18243.071995] Call Trace:
[18243.075369] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.081572] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18243.088740] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18243.095983] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.102099] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.110103] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.116785] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.124717] migration/15 S 0000000000000000 0 84 2 0x00000000
[18243.133274] ffff8804b97bfe80 0000000000000046 ffff8804b976edd0 ffff8804b97bffd8
[18243.142247] ffff8804b97bffd8 ffff8804b97bffd8 ffff8804b976edd0 ffff8804b976edd0
[18243.151210] ffff8804ba74e160 ffffffff81a08f40 0000000000000000 0000000000000000
[18243.160168] Call Trace:
[18243.163527] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.169712] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18243.176855] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18243.184015] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.190103] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.198048] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.204710] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.212630] ksoftirqd/15 S 0000000000000000 0 85 2 0x00000000
[18243.221165] ffff8804b97cbe80 0000000000000046 ffff8804b97c0000 ffff8804b97cbfd8
[18243.230119] ffff8804b97cbfd8 ffff8804b97cbfd8 ffff8804b97c0000 ffff8804b97c0000
[18243.239077] ffff8804ba74e170 ffffffff819f1100 0000000000000000 0000000000000000
[18243.248024] Call Trace:
[18243.251382] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.257574] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18243.264724] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18243.271872] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.277947] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.285870] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.292545] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.300478] kworker/15:0H S ffff88082d9d6840 0 87 2 0x00000000
[18243.309035] ffff8804b97d7e50 0000000000000046 ffff8804b97c1f60 ffff8804b97d7fd8
[18243.317991] ffff8804b97d7fd8 ffff8804b97d7fd8 ffff8804b97c1f60 ffff88082d9d6860
[18243.326942] ffff8804b97d0030 ffff8804b97c1f60 ffff8804b97d0000 ffff88082d9d6840
[18243.335879] Call Trace:
[18243.339244] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.345438] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18243.352309] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18243.359464] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.365555] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.373486] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.380152] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.388068] watchdog/16 S 0000000000000000 0 88 2 0x00000000
[18243.396602] ffff880179757e80 0000000000000046 ffff88017970edd0 ffff880179757fd8
[18243.405554] ffff880179757fd8 ffff880179757fd8 ffff88017970edd0 ffff88017970edd0
[18243.414501] ffff88017fc7cff0 ffffffff81a09820 0000000000000000 0000000000000000
[18243.423457] Call Trace:
[18243.426813] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.433003] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18243.440164] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18243.447329] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.453413] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.461354] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.468017] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.475931] migration/16 S 0000000000000000 0 89 2 0x00000000
[18243.484489] ffff880179783e80 0000000000000046 ffff88017970de20 ffff880179783fd8
[18243.493449] ffff880179783fd8 ffff880179783fd8 ffff88017970de20 ffff88017970de20
[18243.502390] ffff88017fc7cfe0 ffffffff81a08f40 0000000000000000 0000000000000000
[18243.511336] Call Trace:
[18243.514690] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.520864] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18243.528026] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18243.535177] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.541262] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.549189] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.555844] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.563760] ksoftirqd/16 S 0000000000000000 0 90 2 0x00000000
[18243.572286] ffff880179787e80 0000000000000046 ffff88017970ce70 ffff880179787fd8
[18243.581236] ffff880179787fd8 ffff880179787fd8 ffff88017970ce70 ffff88017970ce70
[18243.590182] ffff88017fc7cfd0 ffffffff819f1100 0000000000000000 0000000000000000
[18243.599139] Call Trace:
[18243.602500] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.608675] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18243.615821] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18243.622976] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.629054] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.636978] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.643638] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.651560] kworker/16:0 S ffff88042e216480 0 91 2 0x00000000
[18243.660124] ffff88017978be50 0000000000000046 ffff88017970bec0 ffff88017978bfd8
[18243.669075] ffff88017978bfd8 ffff88017978bfd8 ffff88017970bec0 ffff88042e2164a0
[18243.678034] ffff8804b97d02b0 ffff88017970bec0 ffff8804b97d0280 ffff88042e216480
[18243.686970] Call Trace:
[18243.690347] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.696536] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18243.703398] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18243.710560] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.716646] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.724582] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.731246] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.739171] kworker/16:0H S ffff88042e216840 0 92 2 0x00000000
[18243.747702] ffff88017978fe50 0000000000000046 ffff88017970af10 ffff88017978ffd8
[18243.756651] ffff88017978ffd8 ffff88017978ffd8 ffff88017970af10 ffff88042e216860
[18243.765604] ffff8804b97d03b0 ffff88017970af10 ffff8804b97d0380 ffff88042e216840
[18243.774556] Call Trace:
[18243.777909] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.784094] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18243.790937] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18243.798084] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.804160] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.812093] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.818750] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.826674] watchdog/17 S 0000000000000000 0 93 2 0x00000000
[18243.835213] ffff8801797b3e80 0000000000000046 ffff8801797a8000 ffff8801797b3fd8
[18243.844160] ffff8801797b3fd8 ffff8801797b3fd8 ffff8801797a8000 ffff8801797a8000
[18243.853113] ffff88017fc7cfc0 ffffffff81a09820 0000000000000000 0000000000000000
[18243.862055] Call Trace:
[18243.865416] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.871601] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18243.878745] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18243.885890] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.891955] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.899876] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.906527] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.914446] migration/17 S 0000000000000000 0 94 2 0x00000000
[18243.922971] ffff8801797b7e80 0000000000000046 ffff8801797a8fb0 ffff8801797b7fd8
[18243.931923] ffff8801797b7fd8 ffff8801797b7fd8 ffff8801797a8fb0 ffff8801797a8fb0
[18243.940855] ffff88017fc7cfb0 ffffffff81a08f40 0000000000000000 0000000000000000
[18243.949790] Call Trace:
[18243.953148] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18243.959326] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18243.966497] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18243.973642] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18243.979716] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18243.987646] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18243.994302] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.002227] ksoftirqd/17 S 0000000000000000 0 95 2 0x00000000
[18244.010748] ffff8801797c3e80 0000000000000046 ffff8801797a9f60 ffff8801797c3fd8
[18244.019694] ffff8801797c3fd8 ffff8801797c3fd8 ffff8801797a9f60 ffff8801797a9f60
[18244.028636] ffff88017fc7cfa0 ffffffff819f1100 0000000000000000 0000000000000000
[18244.037581] Call Trace:
[18244.040937] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.047123] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18244.054274] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18244.061440] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.067535] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.075478] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.082143] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.090064] kworker/17:0H S ffff88042e256840 0 97 2 0x00000000
[18244.098624] ffff8801797cbe50 0000000000000046 ffff8801797abec0 ffff8801797cbfd8
[18244.107586] ffff8801797cbfd8 ffff8801797cbfd8 ffff8801797abec0 ffff88042e256860
[18244.116545] ffff8804b97d0730 ffff8801797abec0 ffff8804b97d0700 ffff88042e256840
[18244.125498] Call Trace:
[18244.128852] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.135034] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18244.141892] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18244.149060] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.155154] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.163080] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.169738] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.177661] watchdog/18 S 0000000000000000 0 98 2 0x00000000
[18244.186200] ffff8801797e3e80 0000000000000046 ffff8801797ace70 ffff8801797e3fd8
[18244.195160] ffff8801797e3fd8 ffff8801797e3fd8 ffff8801797ace70 ffff8801797ace70
[18244.204111] ffff88017fc7cf90 ffffffff81a09820 0000000000000000 0000000000000000
[18244.213079] Call Trace:
[18244.216452] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.222642] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18244.229791] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18244.236960] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.243055] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.250980] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.257642] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.265563] migration/18 S 0000000000000000 0 99 2 0x00000000
[18244.274089] ffff8801797e7e80 0000000000000046 ffff8801797ade20 ffff8801797e7fd8
[18244.283039] ffff8801797e7fd8 ffff8801797e7fd8 ffff8801797ade20 ffff8801797ade20
[18244.291983] ffff88017fc7cf80 ffffffff81a08f40 0000000000000000 0000000000000000
[18244.300919] Call Trace:
[18244.304277] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.310473] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18244.317625] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18244.324776] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.330854] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.338787] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.345458] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.353382] ksoftirqd/18 S 0000000000000000 0 100 2 0x00000000
[18244.361914] ffff8801797f3e80 0000000000000046 ffff8801797aedd0 ffff8801797f3fd8
[18244.370873] ffff8801797f3fd8 ffff8801797f3fd8 ffff8801797aedd0 ffff8801797aedd0
[18244.379809] ffff88017fc7cf70 ffffffff819f1100 0000000000000000 0000000000000000
[18244.388743] Call Trace:
[18244.392100] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.398281] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18244.405441] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18244.412593] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.418669] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.426751] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.433431] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.441382] kworker/18:0H S ffff88042e296840 0 102 2 0x00000000
[18244.449926] ffff880179003e50 0000000000000046 ffff8801797f8fb0 ffff880179003fd8
[18244.458887] ffff880179003fd8 ffff880179003fd8 ffff8801797f8fb0 ffff88042e296860
[18244.467855] ffff8804b97d0ab0 ffff8801797f8fb0 ffff8804b97d0a80 ffff88042e296840
[18244.476794] Call Trace:
[18244.480165] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.486357] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18244.493217] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18244.500375] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.506463] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.514398] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.521059] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.528979] watchdog/19 S 0000000000000000 0 103 2 0x00000000
[18244.537508] ffff88017901fe80 0000000000000046 ffff8801797f9f60 ffff88017901ffd8
[18244.546452] ffff88017901ffd8 ffff88017901ffd8 ffff8801797f9f60 ffff8801797f9f60
[18244.555409] ffff88017fc7cf60 ffffffff81a09820 0000000000000000 0000000000000000
[18244.564362] Call Trace:
[18244.567719] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.573898] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18244.581049] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18244.588208] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.594295] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.602229] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.608887] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.616807] migration/19 S 0000000000000000 0 104 2 0x00000000
[18244.625337] ffff880179023e80 0000000000000046 ffff8801797faf10 ffff880179023fd8
[18244.634293] ffff880179023fd8 ffff880179023fd8 ffff8801797faf10 ffff8801797faf10
[18244.643239] ffff88017fc7cf50 ffffffff81a08f40 0000000000000000 0000000000000000
[18244.652191] Call Trace:
[18244.655550] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.661725] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18244.668875] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18244.676023] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.682110] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.690036] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.696694] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.704621] ksoftirqd/19 S 0000000000000000 0 105 2 0x00000000
[18244.713151] ffff880179027e80 0000000000000046 ffff8801797fbec0 ffff880179027fd8
[18244.722109] ffff880179027fd8 ffff880179027fd8 ffff8801797fbec0 ffff8801797fbec0
[18244.731042] ffff88017fc7cf40 ffffffff819f1100 0000000000000000 0000000000000000
[18244.739976] Call Trace:
[18244.743332] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.749510] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18244.756667] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18244.763815] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.769891] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.777813] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.784483] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.792411] kworker/19:0H S ffff88042e2d6840 0 107 2 0x00000000
[18244.800946] ffff88017902fe50 0000000000000046 ffff8801797fde20 ffff88017902ffd8
[18244.809891] ffff88017902ffd8 ffff88017902ffd8 ffff8801797fde20 ffff88042e2d6860
[18244.818842] ffff8804b97d0e30 ffff8801797fde20 ffff8804b97d0e00 ffff88042e2d6840
[18244.827778] Call Trace:
[18244.831135] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.837313] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18244.844179] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18244.851329] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.857408] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.865333] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.871990] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.879909] watchdog/20 S 0000000000000000 0 108 2 0x00000000
[18244.888458] ffff880179037e80 0000000000000046 ffff8801797fedd0 ffff880179037fd8
[18244.897420] ffff880179037fd8 ffff880179037fd8 ffff8801797fedd0 ffff8801797fedd0
[18244.906377] ffff88017fc7cf30 ffffffff81a09820 0000000000000000 0000000000000000
[18244.915333] Call Trace:
[18244.918694] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18244.924869] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18244.932025] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18244.939181] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18244.945273] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.953209] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18244.959867] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18244.967793] migration/20 S 0000000000000000 0 109 2 0x00000000
[18244.976351] ffff88017905be80 0000000000000046 ffff880179050000 ffff88017905bfd8
[18244.985308] ffff88017905bfd8 ffff88017905bfd8 ffff880179050000 ffff880179050000
[18244.994254] ffff88017fc7cf20 ffffffff81a08f40 0000000000000000 0000000000000000
[18245.003207] Call Trace:
[18245.006582] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.012762] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18245.019909] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18245.027072] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.033170] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.041121] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.047778] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.055695] ksoftirqd/20 S 0000000000000000 0 110 2 0x00000000
[18245.064242] ffff88017905fe80 0000000000000046 ffff880179050fb0 ffff88017905ffd8
[18245.073218] ffff88017905ffd8 ffff88017905ffd8 ffff880179050fb0 ffff880179050fb0
[18245.082183] ffff88017fc7cf10 ffffffff819f1100 0000000000000000 0000000000000000
[18245.091137] Call Trace:
[18245.094502] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.100678] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18245.107823] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18245.114969] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.121042] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.128964] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.135632] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.143568] kworker/20:0 S ffff88042e316480 0 111 2 0x00000000
[18245.152118] ffff880179063e50 0000000000000046 ffff880179051f60 ffff880179063fd8
[18245.161077] ffff880179063fd8 ffff880179063fd8 ffff880179051f60 ffff88042e3164a0
[18245.170017] ffff8804b97d10b0 ffff880179051f60 ffff8804b97d1080 ffff88042e316480
[18245.178968] Call Trace:
[18245.182328] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.188506] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18245.195368] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18245.202523] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.208616] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.216544] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.223224] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.231151] kworker/20:0H S ffff88042e316840 0 112 2 0x00000000
[18245.239701] ffff880179067e50 0000000000000046 ffff880179052f10 ffff880179067fd8
[18245.248663] ffff880179067fd8 ffff880179067fd8 ffff880179052f10 ffff88042e316860
[18245.257638] ffff8804b97d11b0 ffff880179052f10 ffff8804b97d1180 ffff88042e316840
[18245.266578] Call Trace:
[18245.269931] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.276116] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18245.282975] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18245.290140] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.296213] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.304144] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.310800] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.318713] watchdog/21 S 0000000000000000 0 113 2 0x00000000
[18245.327255] ffff88017908fe80 0000000000000046 ffff880179053ec0 ffff88017908ffd8
[18245.336211] ffff88017908ffd8 ffff88017908ffd8 ffff880179053ec0 ffff880179053ec0
[18245.345181] ffff88017fc7cf00 ffffffff81a09820 0000000000000000 0000000000000000
[18245.354123] Call Trace:
[18245.357492] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.363678] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18245.370831] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18245.377982] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.384051] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.391971] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.398634] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.406557] migration/21 S 0000000000000000 0 114 2 0x00000000
[18245.415103] ffff8801790abe80 0000000000000046 ffff880179054e70 ffff8801790abfd8
[18245.424065] ffff8801790abfd8 ffff8801790abfd8 ffff880179054e70 ffff880179054e70
[18245.433013] ffff88017fc7cef0 ffffffff81a08f40 0000000000000000 0000000000000000
[18245.441955] Call Trace:
[18245.445326] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.451500] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18245.458658] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18245.465802] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.471876] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.479810] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.486472] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.494393] ksoftirqd/21 S 0000000000000000 0 115 2 0x00000000
[18245.502930] ffff8801790afe80 0000000000000046 ffff880179055e20 ffff8801790affd8
[18245.511880] ffff8801790affd8 ffff8801790affd8 ffff880179055e20 ffff880179055e20
[18245.520833] ffff88017fc7cee0 ffffffff819f1100 0000000000000000 0000000000000000
[18245.529775] Call Trace:
[18245.533138] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.539336] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18245.546489] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18245.553649] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.559724] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.567652] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.574321] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.582243] kworker/21:0H S ffff88042e356840 0 117 2 0x00000000
[18245.590787] ffff8801790b7e50 0000000000000046 ffff8801790b8000 ffff8801790b7fd8
[18245.599719] ffff8801790b7fd8 ffff8801790b7fd8 ffff8801790b8000 ffff88042e356860
[18245.608674] ffff8804b97d1530 ffff8801790b8000 ffff8804b97d1500 ffff88042e356840
[18245.617638] Call Trace:
[18245.620997] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.627186] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18245.634040] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18245.641186] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.647275] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.655208] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.661866] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.669785] watchdog/22 S 0000000000000000 0 118 2 0x00000000
[18245.678327] ffff8801790dbe80 0000000000000046 ffff8801790b8fb0 ffff8801790dbfd8
[18245.687294] ffff8801790dbfd8 ffff8801790dbfd8 ffff8801790b8fb0 ffff8801790b8fb0
[18245.696256] ffff88017fc7ced0 ffffffff81a09820 0000000000000000 0000000000000000
[18245.705200] Call Trace:
[18245.708559] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.714735] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18245.721882] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18245.729036] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.735118] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.743056] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.749720] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.757656] migration/22 S 0000000000000000 0 119 2 0x00000000
[18245.766199] ffff8801790dfe80 0000000000000046 ffff8801790b9f60 ffff8801790dffd8
[18245.775149] ffff8801790dffd8 ffff8801790dffd8 ffff8801790b9f60 ffff8801790b9f60
[18245.784096] ffff88017fc7cec0 ffffffff81a08f40 0000000000000000 0000000000000000
[18245.793035] Call Trace:
[18245.796406] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.802599] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18245.809744] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18245.816893] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.822976] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.830898] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.837572] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.845504] ksoftirqd/22 S 0000000000000000 0 120 2 0x00000000
[18245.854032] ffff8801790e3e80 0000000000000046 ffff8801790baf10 ffff8801790e3fd8
[18245.862985] ffff8801790e3fd8 ffff8801790e3fd8 ffff8801790baf10 ffff8801790baf10
[18245.871930] ffff88017fc7ceb0 ffffffff819f1100 0000000000000000 0000000000000000
[18245.880871] Call Trace:
[18245.884241] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.890417] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18245.897562] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18245.904716] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.910790] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.918718] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18245.925391] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18245.933316] kworker/22:0H S ffff88042e396840 0 122 2 0x00000000
[18245.941857] ffff8801790ebe50 0000000000000046 ffff8801790bce70 ffff8801790ebfd8
[18245.950816] ffff8801790ebfd8 ffff8801790ebfd8 ffff8801790bce70 ffff88042e396860
[18245.959774] ffff8804b97d18b0 ffff8801790bce70 ffff8804b97d1880 ffff88042e396840
[18245.968703] Call Trace:
[18245.972060] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18245.978252] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18245.985128] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18245.992283] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18245.998366] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.006298] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.012957] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.020889] watchdog/23 S 0000000000000000 0 123 2 0x00000000
[18246.029423] ffff880179103e80 0000000000000046 ffff8801790bde20 ffff880179103fd8
[18246.038397] ffff880179103fd8 ffff880179103fd8 ffff8801790bde20 ffff8801790bde20
[18246.047363] ffff88017fc7cea0 ffffffff81a09820 0000000000000000 0000000000000000
[18246.056312] Call Trace:
[18246.059684] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.065861] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.073009] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.080172] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.086257] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.094188] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.100853] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.108780] migration/23 S 0000000000000000 0 124 2 0x00000000
[18246.117335] ffff880179113e80 0000000000000046 ffff8801790bedd0 ffff880179113fd8
[18246.126310] ffff880179113fd8 ffff880179113fd8 ffff8801790bedd0 ffff8801790bedd0
[18246.135277] ffff88017fc7ce90 ffffffff81a08f40 0000000000000000 0000000000000000
[18246.144245] Call Trace:
[18246.147613] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.153795] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.160944] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.168105] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.174186] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.182125] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.188785] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.196720] ksoftirqd/23 S 0000000000000000 0 125 2 0x00000000
[18246.205269] ffff880179117e80 0000000000000046 ffff880179118000 ffff880179117fd8
[18246.214227] ffff880179117fd8 ffff880179117fd8 ffff880179118000 ffff880179118000
[18246.223183] ffff88017fc7ce80 ffffffff819f1100 0000000000000000 0000000000000000
[18246.232125] Call Trace:
[18246.235488] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.241669] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.248816] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.255963] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.262039] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.269963] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.276637] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.284577] kworker/23:0H S ffff88042e3d6840 0 127 2 0x00000000
[18246.293126] ffff880179127e50 0000000000000046 ffff880179119f60 ffff880179127fd8
[18246.302080] ffff880179127fd8 ffff880179127fd8 ffff880179119f60 ffff88042e3d6860
[18246.311050] ffff8804b97d1c30 ffff880179119f60 ffff8804b97d1c00 ffff88042e3d6840
[18246.320001] Call Trace:
[18246.323369] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.329544] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18246.336423] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18246.343577] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.349667] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.357596] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.364264] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.372187] watchdog/24 R running task 0 128 2 0x00000000
[18246.380734] ffff8804b97dfe80 0000000000000046 ffff8804b97c6dd0 ffff8804b97dffd8
[18246.389699] ffff8804b97dffd8 ffff8804b97dffd8 ffff8804b97c6dd0 ffff8804b97c6dd0
[18246.398661] ffff8804ba74eff0 ffffffff81a09820 0000000000000000 0000000000000000
[18246.407615] Call Trace:
[18246.410973] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.417159] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.424325] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.431484] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.437566] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.445499] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.452164] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.460105] migration/24 R running task 0 129 2 0x00000000
[18246.468668] ffff8804b9003e80 0000000000000046 ffff8804b97c5e20 ffff8804b9003fd8
[18246.477634] ffff8804b9003fd8 ffff8804b9003fd8 ffff8804b97c5e20 ffff8804b97c5e20
[18246.486577] ffff8804ba74efe0 ffffffff81a08f40 0000000000000000 0000000000000000
[18246.495542] Call Trace:
[18246.498908] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.505092] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.512271] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.519455] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.525545] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.533482] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.540160] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.548075] ksoftirqd/24 R running task 0 130 2 0x00000000
[18246.556627] ffff8804b9007e80 0000000000000046 ffff8804b97c4e70 ffff8804b9007fd8
[18246.565595] ffff8804b9007fd8 ffff8804b9007fd8 ffff8804b97c4e70 ffff8804b97c4e70
[18246.574554] ffff8804ba74efd0 ffffffff819f1100 0000000000000000 0000000000000000
[18246.583521] Call Trace:
[18246.586895] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.593088] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.600254] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.607421] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.613522] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.621473] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.628151] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.636093] kworker/24:0H S ffff88082da16840 0 132 2 0x00000000
[18246.644647] ffff8804b900fe50 0000000000000046 ffff8804b97c2f10 ffff8804b900ffd8
[18246.653618] ffff8804b900ffd8 ffff8804b900ffd8 ffff8804b97c2f10 ffff88082da16860
[18246.662589] ffff8804b97d1fb0 ffff8804b97c2f10 ffff8804b97d1f80 ffff88082da16840
[18246.671549] Call Trace:
[18246.674916] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.681111] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18246.687990] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18246.695158] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.701258] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.709206] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.715882] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.723825] watchdog/25 S 0000000000000000 0 133 2 0x00000000
[18246.732364] ffff8804b9037e80 0000000000000046 ffff8804b9048000 ffff8804b9037fd8
[18246.741324] ffff8804b9037fd8 ffff8804b9037fd8 ffff8804b9048000 ffff8804b9048000
[18246.750286] ffff8804ba74efc0 ffffffff81a09820 0000000000000000 0000000000000000
[18246.759243] Call Trace:
[18246.762602] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.768781] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.775939] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.783097] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.789186] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.797112] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.803773] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.811695] migration/25 S 0000000000000000 0 134 2 0x00000000
[18246.820223] ffff8804b9053e80 0000000000000046 ffff8804b9048fb0 ffff8804b9053fd8
[18246.829184] ffff8804b9053fd8 ffff8804b9053fd8 ffff8804b9048fb0 ffff8804b9048fb0
[18246.838130] ffff8804ba74efb0 ffffffff81a08f40 0000000000000000 0000000000000000
[18246.847074] Call Trace:
[18246.850430] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.856617] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.863785] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.870930] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.877011] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.885303] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.892323] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.900256] ksoftirqd/25 S 0000000000000000 0 135 2 0x00000000
[18246.908805] ffff8804b9057e80 0000000000000046 ffff8804b9049f60 ffff8804b9057fd8
[18246.917768] ffff8804b9057fd8 ffff8804b9057fd8 ffff8804b9049f60 ffff8804b9049f60
[18246.926724] ffff8804ba74efa0 ffffffff819f1100 0000000000000000 0000000000000000
[18246.935665] Call Trace:
[18246.939020] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18246.945194] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18246.952348] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18246.959501] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18246.965583] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.973526] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18246.980190] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18246.988120] kworker/25:0H S ffff88082da56840 0 137 2 0x00000000
[18246.996652] ffff8804b9067e50 0000000000000046 ffff8804b904bec0 ffff8804b9067fd8
[18247.005615] ffff8804b9067fd8 ffff8804b9067fd8 ffff8804b904bec0 ffff88082da56860
[18247.014569] ffff880179134330 ffff8804b904bec0 ffff880179134300 ffff88082da56840
[18247.023511] Call Trace:
[18247.026863] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.033047] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18247.039902] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18247.047047] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.053122] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.061045] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.067713] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.075631] watchdog/26 S 0000000000000000 0 138 2 0x00000000
[18247.084180] ffff8804b9083e80 0000000000000046 ffff8804b904ce70 ffff8804b9083fd8
[18247.093137] ffff8804b9083fd8 ffff8804b9083fd8 ffff8804b904ce70 ffff8804b904ce70
[18247.102077] ffff8804ba74ef90 ffffffff81a09820 0000000000000000 0000000000000000
[18247.111016] Call Trace:
[18247.114379] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.120569] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18247.127733] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18247.134883] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.140958] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.148882] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.155550] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.163479] migration/26 S 0000000000000000 0 139 2 0x00000000
[18247.172016] ffff8804b9087e80 0000000000000046 ffff8804b904de20 ffff8804b9087fd8
[18247.180968] ffff8804b9087fd8 ffff8804b9087fd8 ffff8804b904de20 ffff8804b904de20
[18247.189913] ffff8804ba74ef80 ffffffff81a08f40 0000000000000000 0000000000000000
[18247.198853] Call Trace:
[18247.202210] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.208396] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18247.215554] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18247.222711] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.228788] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.236720] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.243389] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.251314] ksoftirqd/26 S 0000000000000000 0 140 2 0x00000000
[18247.259847] ffff8804b908be80 0000000000000046 ffff8804b904edd0 ffff8804b908bfd8
[18247.268789] ffff8804b908bfd8 ffff8804b908bfd8 ffff8804b904edd0 ffff8804b904edd0
[18247.277740] ffff8804ba74ef70 ffffffff819f1100 0000000000000000 0000000000000000
[18247.286688] Call Trace:
[18247.290048] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.296225] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18247.303375] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18247.310523] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.316615] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.324896] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.331934] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.339833] kworker/26:0H S ffff88082da96840 0 142 2 0x00000000
[18247.348387] ffff8804b90a3e50 0000000000000046 ffff8804b9090fb0 ffff8804b90a3fd8
[18247.357354] ffff8804b90a3fd8 ffff8804b90a3fd8 ffff8804b9090fb0 ffff88082da96860
[18247.366321] ffff8801791346b0 ffff8804b9090fb0 ffff880179134680 ffff88082da96840
[18247.375270] Call Trace:
[18247.378633] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.384812] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18247.391668] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18247.398822] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.404906] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.412828] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.419495] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.427427] watchdog/27 S 0000000000000000 0 143 2 0x00000000
[18247.435955] ffff8804b90bfe80 0000000000000046 ffff8804b9091f60 ffff8804b90bffd8
[18247.444897] ffff8804b90bffd8 ffff8804b90bffd8 ffff8804b9091f60 ffff8804b9091f60
[18247.453829] ffff8804ba74ef60 ffffffff81a09820 0000000000000000 0000000000000000
[18247.462758] Call Trace:
[18247.466111] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.472290] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18247.479453] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18247.486611] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.492696] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.500630] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.507296] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.515218] migration/27 S 0000000000000000 0 144 2 0x00000000
[18247.523748] ffff8804b90c3e80 0000000000000046 ffff8804b9092f10 ffff8804b90c3fd8
[18247.532702] ffff8804b90c3fd8 ffff8804b90c3fd8 ffff8804b9092f10 ffff8804b9092f10
[18247.541647] ffff8804ba74ef50 ffffffff81a08f40 0000000000000000 0000000000000000
[18247.550602] Call Trace:
[18247.553964] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.560154] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18247.567319] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18247.574472] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.580559] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.588499] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.595159] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.603065] ksoftirqd/27 S 0000000000000000 0 145 2 0x00000000
[18247.611595] ffff8804b90c7e80 0000000000000046 ffff8804b9093ec0 ffff8804b90c7fd8
[18247.620546] ffff8804b90c7fd8 ffff8804b90c7fd8 ffff8804b9093ec0 ffff8804b9093ec0
[18247.629506] ffff8804ba74ef40 ffffffff819f1100 0000000000000000 0000000000000000
[18247.638447] Call Trace:
[18247.641805] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.647982] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18247.655129] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18247.662304] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.668395] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.676333] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.682989] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.690911] kworker/27:0 S ffff88082dad6480 0 146 2 0x00000000
[18247.699447] ffff8804b90cbe50 0000000000000046 ffff8804b9094e70 ffff8804b90cbfd8
[18247.708411] ffff8804b90cbfd8 ffff8804b90cbfd8 ffff8804b9094e70 ffff88082dad64a0
[18247.717364] ffff880179134930 ffff8804b9094e70 ffff880179134900 ffff88082dad6480
[18247.726318] Call Trace:
[18247.729672] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.735839] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18247.742703] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18247.749851] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.755926] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.763853] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.770890] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.779178] kworker/27:0H S ffff88082dad6840 0 147 2 0x00000000
[18247.787750] ffff8804b90cfe50 0000000000000046 ffff8804b9095e20 ffff8804b90cffd8
[18247.796725] ffff8804b90cffd8 ffff8804b90cffd8 ffff8804b9095e20 ffff88082dad6860
[18247.805676] ffff880179134a30 ffff8804b9095e20 ffff880179134a00 ffff88082dad6840
[18247.814619] Call Trace:
[18247.817973] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.824136] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18247.831005] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18247.838152] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.844226] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.852146] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.858792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.866716] watchdog/28 R running task 0 148 2 0x00000000
[18247.875249] ffff8804b90d7e80 0000000000000046 ffff8804b9096dd0 ffff8804b90d7fd8
[18247.884213] ffff8804b90d7fd8 ffff8804b90d7fd8 ffff8804b9096dd0 ffff8804b9096dd0
[18247.893159] ffff8804ba74ef30 ffffffff81a09820 0000000000000000 0000000000000000
[18247.902099] Call Trace:
[18247.905454] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.911635] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18247.918797] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18247.925938] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18247.932013] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.939936] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18247.946597] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18247.954533] migration/28 R running task 0 149 2 0x00000000
[18247.963059] ffff8804b90f3e80 0000000000000046 ffff8804b90e8000 ffff8804b90f3fd8
[18247.972014] ffff8804b90f3fd8 ffff8804b90f3fd8 ffff8804b90e8000 ffff8804b90e8000
[18247.980976] ffff8804ba74ef20 ffffffff81a08f40 0000000000000000 0000000000000000
[18247.989918] Call Trace:
[18247.993296] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18247.999485] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18248.006651] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18248.013805] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.019886] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.027814] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.034484] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.042415] ksoftirqd/28 R running task 0 150 2 0x00000000
[18248.050937] ffff8804b90f7e80 0000000000000046 ffff8804b90e8fb0 ffff8804b90f7fd8
[18248.059895] ffff8804b90f7fd8 ffff8804b90f7fd8 ffff8804b90e8fb0 ffff8804b90e8fb0
[18248.068851] ffff8804ba74ef10 ffffffff819f1100 0000000000000000 0000000000000000
[18248.077810] Call Trace:
[18248.081175] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.087374] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18248.094541] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18248.101702] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.107798] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.115741] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.122425] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.130353] kworker/28:0 S ffff88082db16480 0 151 2 0x00000000
[18248.138890] ffff8804b90fbe50 0000000000000046 ffff8804b90e9f60 ffff8804b90fbfd8
[18248.147857] ffff8804b90fbfd8 ffff8804b90fbfd8 ffff8804b90e9f60 ffff88082db164a0
[18248.156806] ffff880179134cb0 ffff8804b90e9f60 ffff880179134c80 ffff88082db16480
[18248.165775] Call Trace:
[18248.169140] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.175343] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18248.182216] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18248.189737] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.196192] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.204107] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.210804] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.218741] kworker/28:0H S ffff88082db16840 0 152 2 0x00000000
[18248.227295] ffff8804b90ffe50 0000000000000046 ffff8804b90eaf10 ffff8804b90fffd8
[18248.236260] ffff8804b90fffd8 ffff8804b90fffd8 ffff8804b90eaf10 ffff88082db16860
[18248.245214] ffff880179134db0 ffff8804b90eaf10 ffff880179134d80 ffff88082db16840
[18248.254160] Call Trace:
[18248.257536] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.263725] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18248.270595] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18248.277755] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.283837] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.291777] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.298450] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.306373] watchdog/29 S 0000000000000000 0 153 2 0x00000000
[18248.314900] ffff8804b911be80 0000000000000046 ffff8804b90ebec0 ffff8804b911bfd8
[18248.323852] ffff8804b911bfd8 ffff8804b911bfd8 ffff8804b90ebec0 ffff8804b90ebec0
[18248.332788] ffff8804ba74ef00 ffffffff81a09820 0000000000000000 0000000000000000
[18248.341733] Call Trace:
[18248.345088] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.351323] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18248.358464] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18248.365631] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.371717] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.379649] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.386320] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.394246] migration/29 S 0000000000000000 0 154 2 0x00000000
[18248.402779] ffff8804b911fe80 0000000000000046 ffff8804b90ece70 ffff8804b911ffd8
[18248.411737] ffff8804b911ffd8 ffff8804b911ffd8 ffff8804b90ece70 ffff8804b90ece70
[18248.420674] ffff8804ba74eef0 ffffffff81a08f40 0000000000000000 0000000000000000
[18248.429619] Call Trace:
[18248.432970] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.439145] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18248.446294] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18248.453442] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.459526] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.467464] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.474121] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.482037] ksoftirqd/29 S 0000000000000000 0 155 2 0x00000000
[18248.490568] ffff8804b9123e80 0000000000000046 ffff8804b90ede20 ffff8804b9123fd8
[18248.499524] ffff8804b9123fd8 ffff8804b9123fd8 ffff8804b90ede20 ffff8804b90ede20
[18248.508480] ffff8804ba74eee0 ffffffff819f1100 0000000000000000 0000000000000000
[18248.517426] Call Trace:
[18248.520798] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.526982] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18248.534131] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18248.541298] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.547386] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.555320] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.562350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.570548] kworker/29:0H S ffff88082db56840 0 157 2 0x00000000
[18248.579079] ffff8804b9133e50 0000000000000046 ffff8804b9128000 ffff8804b9133fd8
[18248.588039] ffff8804b9133fd8 ffff8804b9133fd8 ffff8804b9128000 ffff88082db56860
[18248.596992] ffff880179135130 ffff8804b9128000 ffff880179135100 ffff88082db56840
[18248.606295] Call Trace:
[18248.609658] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.615836] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18248.622705] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18248.630224] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.636324] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.644613] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.651293] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.659208] watchdog/30 S 0000000000000000 0 158 2 0x00000000
[18248.667773] ffff8804b916be80 0000000000000046 ffff8804b9128fb0 ffff8804b916bfd8
[18248.677076] ffff8804b916bfd8 ffff8804b916bfd8 ffff8804b9128fb0 ffff8804b9128fb0
[18248.686036] ffff8804ba74eed0 ffffffff81a09820 0000000000000000 0000000000000000
[18248.694986] Call Trace:
[18248.698372] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.704576] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18248.711742] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18248.719251] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.725345] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.733304] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.739964] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.747881] migration/30 S 0000000000000000 0 159 2 0x00000000
[18248.756756] ffff8804b916fe80 0000000000000046 ffff8804b9129f60 ffff8804b916ffd8
[18248.765740] ffff8804b916ffd8 ffff8804b916ffd8 ffff8804b9129f60 ffff8804b9129f60
[18248.774720] ffff8804ba74eec0 ffffffff81a08f40 0000000000000000 0000000000000000
[18248.783686] Call Trace:
[18248.787413] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.793951] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18248.801082] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18248.808243] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.814343] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.822276] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.829312] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.837255] ksoftirqd/30 S 0000000000000000 0 160 2 0x00000000
[18248.845801] ffff8804b9173e80 0000000000000046 ffff8804b912af10 ffff8804b9173fd8
[18248.854769] ffff8804b9173fd8 ffff8804b9173fd8 ffff8804b912af10 ffff8804b912af10
[18248.863724] ffff8804ba74eeb0 ffffffff819f1100 0000000000000000 0000000000000000
[18248.873009] Call Trace:
[18248.876379] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.882561] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18248.889724] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18248.896880] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.903039] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.911326] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18248.917971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.925868] kworker/30:0 S ffff88082db96480 0 161 2 0x00000000
[18248.934418] ffff8804b9177e50 0000000000000046 ffff8804b912bec0 ffff8804b9177fd8
[18248.943394] ffff8804b9177fd8 ffff8804b9177fd8 ffff8804b912bec0 ffff88082db964a0
[18248.952714] ffff8801791353b0 ffff8804b912bec0 ffff880179135380 ffff88082db96480
[18248.961680] Call Trace:
[18248.965042] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18248.971222] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18248.978078] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18248.985600] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18248.991863] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18248.999790] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.006469] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.014397] kworker/30:0H S ffff88082db96840 0 162 2 0x00000000
[18249.023288] ffff8804b917be50 0000000000000046 ffff8804b912ce70 ffff8804b917bfd8
[18249.032263] ffff8804b917bfd8 ffff8804b917bfd8 ffff8804b912ce70 ffff88082db96860
[18249.041228] ffff8801791354b0 ffff8804b912ce70 ffff880179135480 ffff88082db96840
[18249.050166] Call Trace:
[18249.053528] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.060061] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18249.067255] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18249.074421] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.080500] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.088417] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.095077] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.103348] watchdog/31 S 0000000000000000 0 163 2 0x00000000
[18249.111879] ffff8804b9197e80 0000000000000046 ffff8804b912de20 ffff8804b9197fd8
[18249.120838] ffff8804b9197fd8 ffff8804b9197fd8 ffff8804b912de20 ffff8804b912de20
[18249.129800] ffff8804ba74eea0 ffffffff81a09820 0000000000000000 0000000000000000
[18249.139090] Call Trace:
[18249.142790] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.148948] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18249.156100] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18249.163249] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.169685] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.178016] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.185043] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.192970] migration/31 S 0000000000000000 0 164 2 0x00000000
[18249.201516] ffff8804b919be80 0000000000000046 ffff8804b912edd0 ffff8804b919bfd8
[18249.210561] ffff8804b919bfd8 ffff8804b919bfd8 ffff8804b912edd0 ffff8804b912edd0
[18249.219528] ffff8804ba74ee90 ffffffff81a08f40 0000000000000000 0000000000000000
[18249.228825] Call Trace:
[18249.232183] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.238381] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18249.245541] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18249.252698] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.259152] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.267434] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.274074] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.281992] ksoftirqd/31 S 0000000000000000 0 165 2 0x00000000
[18249.290534] ffff8804b919fe80 0000000000000046 ffff8804b91a0000 ffff8804b919ffd8
[18249.299496] ffff8804b919ffd8 ffff8804b919ffd8 ffff8804b91a0000 ffff8804b91a0000
[18249.308772] ffff8804ba74ee80 ffffffff819f1100 0000000000000000 0000000000000000
[18249.317731] Call Trace:
[18249.321087] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.327278] [<ffffffff810b956d>] smpboot_thread_fn+0xdd/0x180
[18249.334432] [<ffffffff810b9490>] ? lg_double_unlock+0x90/0x90
[18249.341941] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.348248] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.356175] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.362835] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.370763] kworker/31:0H S ffff88082dbd6840 0 167 2 0x00000000
[18249.379480] ffff8804b91afe50 0000000000000046 ffff8804b91a1f60 ffff8804b91affd8
[18249.388828] ffff8804b91affd8 ffff8804b91affd8 ffff8804b91a1f60 ffff88082dbd6860
[18249.397807] ffff880179135830 ffff8804b91a1f60 ffff880179135800 ffff88082dbd6840
[18249.406747] Call Trace:
[18249.410105] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.416298] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18249.423515] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18249.430689] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.436794] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.444737] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.451400] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.459322] kdevtmpfs S 0000000000000000 0 170 2 0x00000000
[18249.468186] ffff88042fde7e70 0000000000000046 ffff88017911ce70 ffff88042fde7fd8
[18249.477139] ffff88042fde7fd8 ffff88042fde7fd8 ffff88017911ce70 0000000000000000
[18249.486085] 0000000000000000 ffff88017911ce70 0000000000000000 0000000000000000
[18249.495033] Call Trace:
[18249.498397] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.504938] [<ffffffff81433f38>] devtmpfsd+0x168/0x180
[18249.511415] [<ffffffff81433dd0>] ? handle_create.isra.2+0x220/0x220
[18249.519151] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.525230] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.533164] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.539834] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.548110] netns S ffff88083fe98200 0 171 2 0x00000000
[18249.556674] ffff880179227e08 0000000000000046 ffff88017911bec0 ffff880179227fd8
[18249.565643] ffff880179227fd8 ffff880179227fd8 ffff88017911bec0 ffff88017911bec0
[18249.574601] ffff88083fe98200 ffff88082f5e8a80 ffff88083fe98230 ffff88083fe98200
[18249.583569] Call Trace:
[18249.587239] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.593790] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18249.600764] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18249.607545] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18249.615491] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.621582] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.629519] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.636542] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.644469] khungtaskd S 000000000000f888 0 172 2 0x00000000
[18249.653012] ffff8801791fbd90 0000000000000046 ffff88017911af10 ffff8801791fbfd8
[18249.661969] ffff8801791fbfd8 ffff8801791fbfd8 ffff88017911af10 ffff8804b9740000
[18249.670919] ffff8801791fbdc0 0000000101139756 ffff8804b9740000 000000000000f888
[18249.680244] Call Trace:
[18249.683963] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.690177] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18249.697360] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18249.704722] [<ffffffff81689ece>] schedule_timeout_interruptible+0x1e/0x20
[18249.713412] [<ffffffff8112ecbc>] watchdog+0x4c/0x2c0
[18249.719829] [<ffffffff8112ec70>] ? reset_hung_task_detector+0x20/0x20
[18249.728122] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.734527] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.742464] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.749130] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.757055] writeback S ffff88082f023200 0 173 2 0x00000000
[18249.765604] ffff88042fd67e08 0000000000000046 ffff88042fda8000 ffff88042fd67fd8
[18249.774947] ffff88042fd67fd8 ffff88042fd67fd8 ffff88042fda8000 ffff88042fda8000
[18249.783909] ffff88082f023200 ffff88082f5ee280 ffff88082f023230 ffff88082f023200
[18249.792863] Call Trace:
[18249.796239] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.802449] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18249.809417] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18249.816532] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18249.824475] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.830570] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.838507] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.845164] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.853084] kintegrityd S ffff88082f023280 0 174 2 0x00000000
[18249.861998] ffff88042fd27e08 0000000000000046 ffff88042fda8fb0 ffff88042fd27fd8
[18249.870949] ffff88042fd27fd8 ffff88042fd27fd8 ffff88042fda8fb0 ffff88042fda8fb0
[18249.879933] ffff88082f023280 ffff88082f5ee480 ffff88082f0232b0 ffff88082f023280
[18249.888880] Call Trace:
[18249.892258] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.898455] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18249.905772] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18249.912543] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18249.920482] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18249.926583] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.934500] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18249.941155] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18249.949470] bioset S ffff88082f023380 0 175 2 0x00000000
[18249.958381] ffff88042fce7e08 0000000000000046 ffff88042fda9f60 ffff88042fce7fd8
[18249.967731] ffff88042fce7fd8 ffff88042fce7fd8 ffff88042fda9f60 ffff88042fda9f60
[18249.977070] ffff88082f023380 ffff88082f5ee680 ffff88082f0233b0 ffff88082f023380
[18249.986409] Call Trace:
[18249.990153] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18249.996717] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18250.003698] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18250.010480] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18250.018424] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.024511] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.032431] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.039441] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.047384] kblockd S ffff88082f023400 0 176 2 0x00000000
[18250.055936] ffff88042fca7e08 0000000000000046 ffff88042fdaaf10 ffff88042fca7fd8
[18250.064908] ffff88042fca7fd8 ffff88042fca7fd8 ffff88042fdaaf10 ffff88042fdaaf10
[18250.073857] ffff88082f023400 ffff88082f5eea80 ffff88082f023430 ffff88082f023400
[18250.082826] Call Trace:
[18250.086190] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.092375] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18250.099342] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18250.106103] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18250.114024] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.120106] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.128023] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.135045] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.142943] md S ffff88082f534380 0 178 2 0x00000000
[18250.151495] ffff88042f9fbe08 0000000000000046 ffff88042fdace70 ffff88042f9fbfd8
[18250.160462] ffff88042f9fbfd8 ffff88042f9fbfd8 ffff88042fdace70 ffff88042fdace70
[18250.169747] ffff88082f534380 ffff88042f9f2480 ffff88082f5343b0 ffff88082f534380
[18250.178840] Call Trace:
[18250.182189] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.188393] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18250.195375] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18250.202136] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18250.210443] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.216866] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.224797] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.231478] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.239407] kworker/19:1 S ffff88042e2d6480 0 180 2 0x00000000
[18250.247956] ffff88042ec3fe50 0000000000000046 ffff88042fdaedd0 ffff88042ec3ffd8
[18250.257273] ffff88042ec3ffd8 ffff88042ec3ffd8 ffff88042fdaedd0 ffff88042e2d64a0
[18250.266229] ffff88042f03a430 ffff88042fdaedd0 ffff88042f03a400 ffff88042e2d6480
[18250.275193] Call Trace:
[18250.278921] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.285460] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18250.292338] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18250.299853] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.305907] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.313831] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.320500] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.328422] kswapd0 S 0000000000000000 0 185 2 0x00000000
[18250.337297] ffff88042c4cbe08 0000000000000046 ffff88042cb03ec0 ffff88042c4cbfd8
[18250.346263] ffff88042c4cbfd8 ffff88042c4cbfd8 ffff88042cb03ec0 ffff88043ffd9000
[18250.355208] ffff88043ffff4e8 0000000000000000 ffff88042c4cbe80 0000000000000000
[18250.364150] Call Trace:
[18250.367513] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.374065] [<ffffffff811989e4>] kswapd+0x394/0x450
[18250.380566] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18250.387721] [<ffffffff81198650>] ? balance_pgdat+0x5e0/0x5e0
[18250.394781] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.400849] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.408763] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.415761] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.423694] kswapd1 S 0000000000000000 0 186 2 0x00000000
[18250.432216] ffff88042c4cfe08 0000000000000046 ffff88042cb04e70 ffff88042c4cffd8
[18250.441150] ffff88042c4cffd8 ffff88042c4cffd8 ffff88042cb04e70 ffff88083ffd6000
[18250.450471] ffff88083fffc4e8 0000000000000000 ffff88042c4cfe80 0000000000000000
[18250.459437] Call Trace:
[18250.462809] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.468982] [<ffffffff811989e4>] kswapd+0x394/0x450
[18250.475153] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18250.482303] [<ffffffff81198650>] ? balance_pgdat+0x5e0/0x5e0
[18250.489713] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.496107] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.503990] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.510640] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.518556] ksmd S 0000000000000000 0 187 2 0x00000000
[18250.527393] ffff88042cb0be50 0000000000000046 ffff88042cb05e20 ffff88042cb0bfd8
[18250.536418] ffff88042cb0bfd8 ffff88042cb0bfd8 ffff88042cb05e20 ffff88042cb05e20
[18250.545358] ffff88042cb05e20 ffff88042cb0be88 ffff88042cb05e20 0000000000000000
[18250.554308] Call Trace:
[18250.557655] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.563842] [<ffffffff811d801a>] ksm_scan_thread+0x16a/0x240
[18250.571234] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18250.578393] [<ffffffff811d7eb0>] ? ksm_do_scan+0x11e0/0x11e0
[18250.585426] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.591496] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.599392] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.606364] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.614438] khugepaged S 000000000000f888 0 188 2 0x00000000
[18250.622919] ffff88042cb13d80 0000000000000046 ffff88042cb06dd0 ffff88042cb13fd8
[18250.631871] ffff88042cb13fd8 ffff88042cb13fd8 ffff88042cb06dd0 ffff8804b96b8000
[18250.640804] ffff88042cb13db0 000000010111f398 ffff8804b96b8000 000000000000f888
[18250.650089] Call Trace:
[18250.653530] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.659698] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18250.666841] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18250.674170] [<ffffffff810b1786>] ? prepare_to_wait+0x56/0x90
[18250.681198] [<ffffffff811e9a13>] khugepaged+0x1e3/0x480
[18250.688078] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18250.695188] [<ffffffff811e9830>] ? khugepaged_scan_mm_slot+0xc70/0xc70
[18250.703185] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.709283] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.717176] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.724149] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.732048] fsnotify_mark S ffff88042c52fe00 0 189 2 0x00000000
[18250.740579] ffff88042c52fe28 0000000000000046 ffff88042cb02f10 ffff88042c52ffd8
[18250.749534] ffff88042c52ffd8 ffff88042c52ffd8 ffff88042cb02f10 ffff88042c52fe58
[18250.758488] ffff88042c52fe58 ffff88042cb02f10 ffff88042bc90258 ffff88042c52fe00
[18250.767757] Call Trace:
[18250.771082] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.777233] [<ffffffff81243811>] fsnotify_mark_destroy+0x141/0x180
[18250.784846] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18250.791972] [<ffffffff812436d0>] ? fsnotify_put_mark+0x40/0x40
[18250.799522] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.805907] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.813818] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.820470] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.828386] crypto S ffff88042c4d0180 0 190 2 0x00000000
[18250.836905] ffff88042c54be08 0000000000000046 ffff88042cb01f60 ffff88042c54bfd8
[18250.845851] ffff88042c54bfd8 ffff88042c54bfd8 ffff88042cb01f60 ffff88042cb01f60
[18250.854842] ffff88042c4d0180 ffff88042cae4680 ffff88042c4d01b0 ffff88042c4d0180
[18250.863788] Call Trace:
[18250.867145] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.873335] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18250.880287] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18250.887386] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18250.895294] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.901365] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.909273] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18250.915920] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18250.924197] kthrotld S ffff88042c4d0280 0 198 2 0x00000000
[18250.932746] ffff88042c573e08 0000000000000046 ffff88042c55ce70 ffff88042c573fd8
[18250.941703] ffff88042c573fd8 ffff88042c573fd8 ffff88042c55ce70 ffff88042c55ce70
[18250.950647] ffff88042c4d0280 ffff88042cae4280 ffff88042c4d02b0 ffff88042c4d0280
[18250.959589] Call Trace:
[18250.962987] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18250.969508] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18250.976464] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18250.983209] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18250.991122] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18250.997178] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.005438] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.012439] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.020367] kworker/8:1 S ffff88082d816480 0 199 2 0x00000000
[18251.028910] ffff88082c8a3e50 0000000000000046 ffff88082c898000 ffff88082c8a3fd8
[18251.037877] ffff88082c8a3fd8 ffff88082c8a3fd8 ffff88082c898000 ffff88082d8164a0
[18251.046834] ffff88082c886030 ffff88082c898000 ffff88082c886000 ffff88082d816480
[18251.056119] Call Trace:
[18251.059487] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.065680] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18251.072541] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18251.079693] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.085767] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.094022] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.100690] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.108615] kmpath_rdacd S ffff88042bd40980 0 203 2 0x00000000
[18251.117141] ffff88042bd6fe08 0000000000000046 ffff88082c89de20 ffff88042bd6ffd8
[18251.126094] ffff88042bd6ffd8 ffff88042bd6ffd8 ffff88082c89de20 ffff88082c89de20
[18251.135398] ffff88042bd40980 ffff88042bd70480 ffff88042bd409b0 ffff88042bd40980
[18251.144349] Call Trace:
[18251.147716] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.153895] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18251.160856] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18251.167618] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18251.175892] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.181971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.189871] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.196527] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.204458] kworker/0:2 S ffff88042e016480 0 204 2 0x00000000
[18251.213345] ffff88042bd8be50 0000000000000046 ffff88042c558000 ffff88042bd8bfd8
[18251.222323] ffff88042bd8bfd8 ffff88042bd8bfd8 ffff88042c558000 ffff88042e0164a0
[18251.231251] ffff88042c4d16b0 ffff88042c558000 ffff88042c4d1680 ffff88042e016480
[18251.240191] Call Trace:
[18251.243557] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.249746] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18251.256982] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18251.264467] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.270553] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.278484] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.285125] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.293038] kworker/15:1 S ffff88082d9d6480 0 205 2 0x00000000
[18251.301913] ffff88082cb1be50 0000000000000046 ffff88082c898fb0 ffff88082cb1bfd8
[18251.310876] ffff88082cb1bfd8 ffff88082cb1bfd8 ffff88082c898fb0 ffff88082d9d64a0
[18251.319822] ffff88082c9d8130 ffff88082c898fb0 ffff88082c9d8100 ffff88082d9d6480
[18251.328776] Call Trace:
[18251.332130] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.338314] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18251.345444] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18251.352947] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.358996] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.366909] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.373570] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.381486] kpsmoused S ffff88082c9d8180 0 206 2 0x00000000
[18251.390368] ffff88042bd8fe08 0000000000000046 ffff88082c899f60 ffff88042bd8ffd8
[18251.399336] ffff88042bd8ffd8 ffff88042bd8ffd8 ffff88082c899f60 ffff88082c899f60
[18251.408280] ffff88082c9d8180 ffff88082cb10080 ffff88082c9d81b0 ffff88082c9d8180
[18251.417221] Call Trace:
[18251.420591] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.426774] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18251.433737] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18251.440494] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18251.448787] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.454887] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.462821] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.469476] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.477389] ipv6_addrconf S ffff88042bde3900 0 207 2 0x00000000
[18251.486234] ffff88042be1be08 0000000000000046 ffff88082c89af10 ffff88042be1bfd8
[18251.495190] ffff88042be1bfd8 ffff88042be1bfd8 ffff88082c89af10 ffff88082c89af10
[18251.504131] ffff88042bde3900 ffff88082c8ace80 ffff88042bde3930 ffff88042bde3900
[18251.513068] Call Trace:
[18251.516436] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.522629] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18251.529581] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18251.536708] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18251.544664] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.550760] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.558690] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.565340] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.573262] deferwq S ffff88042bde3b80 0 226 2 0x00000000
[18251.582180] ffff88042be5fe08 0000000000000046 ffff88042c558fb0 ffff88042be5ffd8
[18251.591111] ffff88042be5ffd8 ffff88042be5ffd8 ffff88042c558fb0 ffff88042c558fb0
[18251.600021] ffff88042bde3b80 ffff88082c8ae880 ffff88042bde3bb0 ffff88042bde3b80
[18251.608979] Call Trace:
[18251.612359] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.618554] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18251.625893] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18251.632859] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18251.640797] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.646883] [<ffffffff810bfde3>] ? finish_task_switch+0x53/0x180
[18251.654323] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.662239] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.668897] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.677162] kworker/1:1 S ffff88042e056480 0 228 2 0x00000000
[18251.685718] ffff88042be77e50 0000000000000046 ffff88042be88000 ffff88042be77fd8
[18251.694688] ffff88042be77fd8 ffff88042be77fd8 ffff88042be88000 ffff88042e0564a0
[18251.703646] ffff880179222430 ffff88042be88000 ffff880179222400 ffff88042e056480
[18251.712602] Call Trace:
[18251.715966] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.722522] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18251.729414] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18251.736573] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.742641] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.750564] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.757216] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.765128] kworker/2:1 S ffff88042e096480 0 229 2 0x00000000
[18251.774012] ffff88042beebe50 0000000000000046 ffff88042be88fb0 ffff88042beebfd8
[18251.782968] ffff88042beebfd8 ffff88042beebfd8 ffff88042be88fb0 ffff88042e0964a0
[18251.791927] ffff88042fce3530 ffff88042be88fb0 ffff88042fce3500 ffff88042e096480
[18251.800891] Call Trace:
[18251.804249] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.810430] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18251.817642] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18251.824793] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.830872] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.838790] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.845441] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.853356] kworker/4:1 S ffff88042e116480 0 230 2 0x00000000
[18251.862243] ffff88042beefe50 0000000000000046 ffff88042be89f60 ffff88042beeffd8
[18251.871255] ffff88042beeffd8 ffff88042beeffd8 ffff88042be89f60 ffff88042e1164a0
[18251.880206] ffff88042c610030 ffff88042be89f60 ffff88042c610000 ffff88042e116480
[18251.889144] Call Trace:
[18251.892508] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.898691] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18251.905916] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18251.913413] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18251.919554] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.927492] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18251.934131] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18251.942068] kworker/6:1 S ffff88042e196480 0 231 2 0x00000000
[18251.950601] ffff88042befbe50 0000000000000046 ffff88042be8af10 ffff88042befbfd8
[18251.959906] ffff88042befbfd8 ffff88042befbfd8 ffff88042be8af10 ffff88042e1964a0
[18251.968895] ffff88082c970030 ffff88042be8af10 ffff88082c970000 ffff88042e196480
[18251.977852] Call Trace:
[18251.981207] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18251.987378] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18251.994225] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.001375] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.007818] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.015907] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.022670] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.030613] kworker/7:1 S ffff88042e1d6480 0 232 2 0x00000000
[18252.039144] ffff88042beffe50 0000000000000046 ffff88042be8bec0 ffff88042befffd8
[18252.048084] ffff88042befffd8 ffff88042befffd8 ffff88042be8bec0 ffff88042e1d64a0
[18252.057377] ffff88042c654030 ffff88042be8bec0 ffff88042c654000 ffff88042e1d6480
[18252.066448] Call Trace:
[18252.069852] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.076030] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.082890] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.090046] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.096116] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.104378] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.111169] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.119083] kworker/13:1 S ffff88082d956480 0 234 2 0x00000000
[18252.127615] ffff88082c4ebe50 0000000000000046 ffff88082c4e0fb0 ffff88082c4ebfd8
[18252.136564] ffff88082c4ebfd8 ffff88082c4ebfd8 ffff88082c4e0fb0 ffff88082d9564a0
[18252.145511] ffff88042c748030 ffff88082c4e0fb0 ffff88042c748000 ffff88082d956480
[18252.154775] Call Trace:
[18252.158487] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.164676] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.171528] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.178686] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.184764] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.192689] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.199344] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.207606] kworker/12:1 S ffff88082d916480 0 235 2 0x00000000
[18252.216188] ffff88082c4efe50 0000000000000046 ffff88082c4e1f60 ffff88082c4effd8
[18252.225138] ffff88082c4effd8 ffff88082c4effd8 ffff88082c4e1f60 ffff88082d9164a0
[18252.234083] ffff88042c720030 ffff88082c4e1f60 ffff88042c720000 ffff88082d916480
[18252.243019] Call Trace:
[18252.246374] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.252918] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.260119] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.267271] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.273350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.281265] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.287922] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.295840] kworker/14:1 S ffff88082d996480 0 236 2 0x00000000
[18252.304593] ffff88082c4f3e50 0000000000000046 ffff88082c4e2f10 ffff88082c4f3fd8
[18252.313867] ffff88082c4f3fd8 ffff88082c4f3fd8 ffff88082c4e2f10 ffff88082d9964a0
[18252.322825] ffff88082c9b0030 ffff88082c4e2f10 ffff88082c9b0000 ffff88082d996480
[18252.331758] Call Trace:
[18252.335106] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.341285] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.348130] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.355276] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.361698] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.369636] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.376289] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.384196] kworker/17:1 S ffff88042e256480 0 237 2 0x00000000
[18252.392728] ffff88042bf03e50 0000000000000046 ffff88042be8edd0 ffff88042bf03fd8
[18252.401671] ffff88042bf03fd8 ffff88042bf03fd8 ffff88042be8edd0 ffff88042e2564a0
[18252.410974] ffff88082ca28030 ffff88042be8edd0 ffff88082ca28000 ffff88042e256480
[18252.419915] Call Trace:
[18252.423251] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.429421] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.436269] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.443420] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.449507] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.457421] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.464421] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.472390] kworker/18:1 S ffff88042e296480 0 239 2 0x00000000
[18252.480919] ffff88042bf0be50 0000000000000046 ffff88042be8ce70 ffff88042bf0bfd8
[18252.489871] ffff88042bf0bfd8 ffff88042bf0bfd8 ffff88042be8ce70 ffff88042e2964a0
[18252.499165] ffff88082ca4a030 ffff88042be8ce70 ffff88082ca4a000 ffff88042e296480
[18252.508088] Call Trace:
[18252.511438] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.517640] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.524487] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.531642] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.537720] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.545649] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.552298] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.560207] kworker/20:1 S ffff88042e316480 0 240 2 0x00000000
[18252.568749] ffff88042bf0fe50 0000000000000046 ffff88042bf10000 ffff88042bf0ffd8
[18252.578058] ffff88042bf0ffd8 ffff88042bf0ffd8 ffff88042bf10000 ffff88042e3164a0
[18252.587027] ffff88042c7b2030 ffff88042bf10000 ffff88042c7b2000 ffff88042e316480
[18252.595960] Call Trace:
[18252.599306] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.605501] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.612599] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.619783] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.625862] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.633782] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.640466] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.648413] kworker/22:1 S ffff88042e396480 0 241 2 0x00000000
[18252.656940] ffff88042bf1be50 0000000000000046 ffff88042bf10fb0 ffff88042bf1bfd8
[18252.665889] ffff88042bf1bfd8 ffff88042bf1bfd8 ffff88042bf10fb0 ffff88042e3964a0
[18252.674843] ffff88042bc04030 ffff88042bf10fb0 ffff88042bc04000 ffff88042e396480
[18252.683775] Call Trace:
[18252.687121] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.693278] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.700105] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.707244] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.713315] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.721227] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.727879] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.735790] kworker/23:1 S ffff88042e3d6480 0 242 2 0x00000000
[18252.744304] ffff88042bf1fe50 0000000000000046 ffff88042bf11f60 ffff88042bf1ffd8
[18252.753254] ffff88042bf1ffd8 ffff88042bf1ffd8 ffff88042bf11f60 ffff88042e3d64a0
[18252.762198] ffff88042bc2c030 ffff88042bf11f60 ffff88042bc2c000 ffff88042e3d6480
[18252.771141] Call Trace:
[18252.774495] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.780663] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.787506] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.794660] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.801082] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.809021] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.815680] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.823598] kworker/21:1 S ffff88042e356480 0 243 2 0x00000000
[18252.832136] ffff88042bf23e50 0000000000000046 ffff88042bf12f10 ffff88042bf23fd8
[18252.841093] ffff88042bf23fd8 ffff88042bf23fd8 ffff88042bf12f10 ffff88042e3564a0
[18252.850048] ffff88042c7da030 ffff88042bf12f10 ffff88042c7da000 ffff88042e356480
[18252.859323] Call Trace:
[18252.862973] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.869121] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.875967] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.883093] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.889156] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.897068] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.903715] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.911969] kworker/25:1 S ffff88082da56480 0 244 2 0x00000000
[18252.920557] ffff88082c4f7e50 0000000000000046 ffff88082c4e6dd0 ffff88082c4f7fd8
[18252.929516] ffff88082c4f7fd8 ffff88082c4f7fd8 ffff88082c4e6dd0 ffff88082da564a0
[18252.938472] ffff88082c8840b0 ffff88082c4e6dd0 ffff88082c884080 ffff88082da56480
[18252.947421] Call Trace:
[18252.950769] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18252.956951] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18252.963804] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18252.971290] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18252.977685] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18252.985608] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18252.992254] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.000168] kworker/9:1 S ffff88082d856480 0 245 2 0x00000000
[18253.008689] ffff88082c4fbe50 0000000000000046 ffff88082c4e5e20 ffff88082c4fbfd8
[18253.017635] ffff88082c4fbfd8 ffff88082c4fbfd8 ffff88082c4e5e20 ffff88082d8564a0
[18253.026951] ffff88042c69c330 ffff88082c4e5e20 ffff88042c69c300 ffff88082d856480
[18253.036225] Call Trace:
[18253.039604] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.045780] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.052625] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.059771] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.065851] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.073781] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.080429] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.088687] kworker/24:1 R running task 0 246 2 0x00000000
[18253.097201] ffff88082c4ffe50 0000000000000046 ffff88082c4e4e70 ffff88082c4fffd8
[18253.106146] ffff88082c4fffd8 ffff88082c4fffd8 ffff88082c4e4e70 ffff88082da164a0
[18253.115089] ffff88042bc56030 ffff88082c4e4e70 ffff88042bc56000 ffff88082da16480
[18253.124008] Call Trace:
[18253.127359] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.133531] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.140770] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.148239] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.154309] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.162223] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.168873] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.176792] kworker/27:1 S ffff88082dad6480 0 247 2 0x00000000
[18253.185314] ffff88082c503e50 0000000000000046 ffff88082c4e3ec0 ffff88082c503fd8
[18253.194291] ffff88082c503fd8 ffff88082c503fd8 ffff88082c4e3ec0 ffff88082dad64a0
[18253.203600] ffff88042bcc6030 ffff88082c4e3ec0 ffff88042bcc6000 ffff88082dad6480
[18253.212559] Call Trace:
[18253.215916] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.222090] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.228956] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.236108] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.242546] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.250485] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.257504] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.265438] kworker/31:1 S ffff88082dbd6480 0 248 2 0x00000000
[18253.273972] ffff88082c507e50 0000000000000046 ffff88082c508000 ffff88082c507fd8
[18253.282926] ffff88082c507fd8 ffff88082c507fd8 ffff88082c508000 ffff88082dbd64a0
[18253.291906] ffff88042bd40a30 ffff88082c508000 ffff88042bd40a00 ffff88082dbd6480
[18253.300849] Call Trace:
[18253.304199] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.310378] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.317587] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.324742] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.330823] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.338749] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.345420] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.353326] kworker/28:1 R running task 0 249 2 0x00000000
[18253.361861] ffff88082c513e50 0000000000000046 ffff88082c508fb0 ffff88082c513fd8
[18253.371160] ffff88082c513fd8 ffff88082c513fd8 ffff88082c508fb0 ffff88082db164a0
[18253.380163] ffff88082ca6a030 ffff88082c508fb0 ffff88082ca6a000 ffff88082db16480
[18253.389096] Call Trace:
[18253.392460] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.398652] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.405508] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.412666] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.418749] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.426673] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.433690] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.441603] kworker/29:1 S ffff88082db56480 0 250 2 0x00000000
[18253.450119] ffff88082c517e50 0000000000000046 ffff88082c509f60 ffff88082c517fd8
[18253.459058] ffff88082c517fd8 ffff88082c517fd8 ffff88082c509f60 ffff88082db564a0
[18253.468002] ffff88082ca8c030 ffff88082c509f60 ffff88082ca8c000 ffff88082db56480
[18253.476939] Call Trace:
[18253.480282] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.486458] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.493653] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.500801] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.506884] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.514825] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.521476] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.529410] kworker/30:1 S ffff88082db96480 0 251 2 0x00000000
[18253.537937] ffff88082c51be50 0000000000000046 ffff88082c50af10 ffff88082c51bfd8
[18253.546870] ffff88082c51bfd8 ffff88082c51bfd8 ffff88082c50af10 ffff88082db964a0
[18253.556172] ffff88042bd16030 ffff88082c50af10 ffff88042bd16000 ffff88082db96480
[18253.565133] Call Trace:
[18253.568494] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.574669] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.581515] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.588660] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.594725] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.602639] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.609367] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.617625] kworker/10:1 S ffff88082d896480 0 252 2 0x00000000
[18253.626116] ffff88082c51fe50 0000000000000046 ffff88082c50bec0 ffff88082c51ffd8
[18253.635062] ffff88082c51ffd8 ffff88082c51ffd8 ffff88082c50bec0 ffff88082d8964a0
[18253.644014] ffff88042c6c6030 ffff88082c50bec0 ffff88042c6c6000 ffff88082d896480
[18253.652941] Call Trace:
[18253.656289] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.662471] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.669313] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.676801] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.683207] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.691100] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.697765] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.705672] kworker/26:1 S ffff88082da96480 0 253 2 0x00000000
[18253.714191] ffff88082c523e50 0000000000000046 ffff88082c50ce70 ffff88082c523fd8
[18253.723137] ffff88082c523fd8 ffff88082c523fd8 ffff88082c50ce70 ffff88082da964a0
[18253.732083] ffff88042bc9e330 ffff88082c50ce70 ffff88042bc9e300 ffff88082da96480
[18253.741392] Call Trace:
[18253.745088] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.751592] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.758466] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.765622] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.771709] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.779637] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.786291] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.794201] kworker/0:3 S ffff88042e016480 0 254 2 0x00000000
[18253.802746] ffff88042bf4be50 0000000000000046 ffff88042bf16dd0 ffff88042bf4bfd8
[18253.811684] ffff88042bf4bfd8 ffff88042bf4bfd8 ffff88042bf16dd0 ffff88042e0164a0
[18253.820639] ffff88042c4d1730 ffff88042bf16dd0 ffff88042c4d1700 ffff88042e016480
[18253.829598] Call Trace:
[18253.832955] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.839125] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.845984] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.853123] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.859191] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.867113] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.873773] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.881695] kworker/5:1 S ffff88042e156480 0 255 2 0x00000000
[18253.890218] ffff88042bf4fe50 0000000000000046 ffff88042bf58000 ffff88042bf4ffd8
[18253.899158] ffff88042bf4ffd8 ffff88042bf4ffd8 ffff88042bf58000 ffff88042e1564a0
[18253.908114] ffff88082c946db0 ffff88042bf58000 ffff88082c946d80 ffff88042e156480
[18253.917047] Call Trace:
[18253.920748] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18253.927268] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18253.934122] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18253.941618] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18253.947934] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.955883] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18253.962557] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18253.970479] kauditd S 0000000000000000 0 367 2 0x00000000
[18253.978992] ffff88042c6bfe48 0000000000000046 ffff88042bf58fb0 ffff88042c6bffd8
[18253.987935] ffff88042c6bffd8 ffff88042c6bffd8 ffff88042bf58fb0 0000000000000000
[18253.996883] ffff88042bf58fb0 ffff88042bf58fb0 ffff88042bf58fb0 0000000000000000
[18254.006160] Call Trace:
[18254.009847] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.016012] [<ffffffff81117794>] kauditd_thread+0x184/0x1e0
[18254.022966] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18254.029824] [<ffffffff81117610>] ? audit_printk_skb+0x70/0x70
[18254.036969] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.043035] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.050973] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.057626] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.065869] ata_sff S ffff88042c69c780 0 452 2 0x00000000
[18254.074418] ffff88042a6d7e08 0000000000000046 ffff88042bf59f60 ffff88042a6d7fd8
[18254.083381] ffff88042a6d7fd8 ffff88042a6d7fd8 ffff88042bf59f60 ffff88042bf59f60
[18254.092320] ffff88042c69c780 ffff88082be84680 ffff88042c69c7b0 ffff88042c69c780
[18254.101247] Call Trace:
[18254.104598] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.110782] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18254.117728] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18254.124821] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18254.133066] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.139124] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.147047] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.153688] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.161609] scsi_eh_0 S 0000000000000000 0 482 2 0x00000000
[18254.170126] ffff880036a9be18 0000000000000046 ffff88042a68af10 ffff880036a9bfd8
[18254.179055] ffff880036a9bfd8 ffff880036a9bfd8 ffff88042a68af10 ffff8801790d0000
[18254.188327] ffff88042a68af10 ffff88042a68b588 0000000000000000 0000000000000000
[18254.197265] Call Trace:
[18254.200641] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.206837] [<ffffffff81457863>] scsi_error_handler+0xa3/0x8b0
[18254.214099] [<ffffffff814577c0>] ? scsi_eh_get_sense+0x250/0x250
[18254.221558] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.227634] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.235582] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.242236] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.250506] scsi_tmf_0 S ffff88042c7db600 0 483 2 0x00000000
[18254.259094] ffff880036a9fe08 0000000000000046 ffff88042a68bec0 ffff880036a9ffd8
[18254.268075] ffff880036a9ffd8 ffff880036a9ffd8 ffff88042a68bec0 ffff88042a68bec0
[18254.277020] ffff88042c7db600 ffff88042bac4480 ffff88042c7db630 ffff88042c7db600
[18254.285969] Call Trace:
[18254.289328] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.295510] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18254.302474] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18254.309373] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18254.317632] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.323719] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.331648] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.338309] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.346346] fw_event_mpt2sa S ffff88042c7db680 0 484 2 0x00000000
[18254.354911] ffff8800361a3e08 0000000000000046 ffff88042a68ce70 ffff8800361a3fd8
[18254.363934] ffff8800361a3fd8 ffff8800361a3fd8 ffff88042a68ce70 ffff88042a68ce70
[18254.373235] ffff88042c7db680 ffff88042bac4680 ffff88042c7db6b0 ffff88042c7db680
[18254.382390] Call Trace:
[18254.385798] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.392007] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18254.398983] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18254.405773] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18254.413726] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.419839] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.427786] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.434815] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.443076] mlx4 S ffff88042c610300 0 490 2 0x00000000
[18254.451634] ffff880429e07e08 0000000000000046 ffff88042a68edd0 ffff880429e07fd8
[18254.460605] ffff880429e07fd8 ffff880429e07fd8 ffff88042a68edd0 ffff88042a68edd0
[18254.469581] ffff88042c610300 ffff88042a5fc480 ffff88042c610330 ffff88042c610300
[18254.478527] Call Trace:
[18254.481912] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.488098] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18254.495067] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18254.502216] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18254.510520] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.516621] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.524549] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.531219] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.539143] mlx4_health S ffff88042c610380 0 491 2 0x00000000
[18254.547698] ffff880429e13e08 0000000000000046 ffff880429e08000 ffff880429e13fd8
[18254.556670] ffff880429e13fd8 ffff880429e13fd8 ffff880429e08000 ffff880429e08000
[18254.565646] ffff88042c610380 ffff88042a5fc280 ffff88042c6103b0 ffff88042c610380
[18254.574967] Call Trace:
[18254.578681] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.584884] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18254.591864] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18254.598645] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18254.606580] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.612662] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.620609] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.627630] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.635565] scsi_eh_1 S 0000000000000000 0 496 2 0x00000000
[18254.644306] ffff880429a4be18 0000000000000046 ffff88042bf13ec0 ffff880429a4bfd8
[18254.653544] ffff880429a4bfd8 ffff880429a4bfd8 ffff88042bf13ec0 ffff8800361de000
[18254.662510] ffff88042bf13ec0 ffff88042bf14538 0000000000000000 0000000000000000
[18254.671465] Call Trace:
[18254.674833] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.681027] [<ffffffff81457863>] scsi_error_handler+0xa3/0x8b0
[18254.688274] [<ffffffff814577c0>] ? scsi_eh_get_sense+0x250/0x250
[18254.695722] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.701815] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.709790] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.716812] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.724768] scsi_tmf_1 S ffff88042c654a80 0 497 2 0x00000000
[18254.733304] ffff880429a4fe08 0000000000000046 ffff880429f00000 ffff880429a4ffd8
[18254.742253] ffff880429a4ffd8 ffff880429a4ffd8 ffff880429f00000 ffff880429f00000
[18254.751221] ffff88042c654a80 ffff88042a432280 ffff88042c654ab0 ffff88042c654a80
[18254.760171] Call Trace:
[18254.763547] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.769733] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18254.776892] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18254.784014] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18254.791944] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.798028] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.805943] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.812614] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.820566] scsi_eh_2 S 0000000000000000 0 498 2 0x00000000
[18254.829103] ffff880429f0fe18 0000000000000046 ffff880429f00fb0 ffff880429f0ffd8
[18254.838051] ffff880429f0ffd8 ffff880429f0ffd8 ffff880429f00fb0 ffff8800361d9800
[18254.847356] ffff880429f00fb0 ffff880429f01628 0000000000000000 0000000000000000
[18254.856327] Call Trace:
[18254.859701] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.865904] [<ffffffff81457863>] scsi_error_handler+0xa3/0x8b0
[18254.873169] [<ffffffff814577c0>] ? scsi_eh_get_sense+0x250/0x250
[18254.880622] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.886707] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.894645] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.901296] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.909590] scsi_tmf_2 S ffff88042c654b80 0 499 2 0x00000000
[18254.918513] ffff880429f1be08 0000000000000046 ffff880429f01f60 ffff880429f1bfd8
[18254.927488] ffff880429f1bfd8 ffff880429f1bfd8 ffff880429f01f60 ffff880429f01f60
[18254.936451] ffff88042c654b80 ffff88042a432880 ffff88042c654bb0 ffff88042c654b80
[18254.945401] Call Trace:
[18254.948783] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18254.954983] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18254.961958] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18254.968723] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18254.976663] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18254.983119] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18254.991364] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18254.998042] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.005959] scsi_eh_3 S 0000000000000000 0 500 2 0x00000000
[18255.014511] ffff880429f1fe18 0000000000000046 ffff880429f02f10 ffff880429f1ffd8
[18255.023464] ffff880429f1ffd8 ffff880429f1ffd8 ffff880429f02f10 ffff8800361de800
[18255.032417] ffff880429f02f10 ffff880429f03588 0000000000000000 0000000000000000
[18255.041360] Call Trace:
[18255.044735] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.051285] [<ffffffff81457863>] scsi_error_handler+0xa3/0x8b0
[18255.058908] [<ffffffff814577c0>] ? scsi_eh_get_sense+0x250/0x250
[18255.066360] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.072458] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.080380] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.087033] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.094963] scsi_tmf_3 S ffff88042c654c80 0 501 2 0x00000000
[18255.103520] ffff880429f23e08 0000000000000046 ffff880429f03ec0 ffff880429f23fd8
[18255.112478] ffff880429f23fd8 ffff880429f23fd8 ffff880429f03ec0 ffff880429f03ec0
[18255.121497] ffff88042c654c80 ffff88042a432e80 ffff88042c654cb0 ffff88042c654c80
[18255.130807] Call Trace:
[18255.134169] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.140351] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18255.147306] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18255.154067] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18255.161994] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.168443] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.176367] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.183041] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.190980] scsi_eh_4 S 0000000000000000 0 502 2 0x00000000
[18255.199537] ffff880429f27e18 0000000000000046 ffff880429f04e70 ffff880429f27fd8
[18255.208521] ffff880429f27fd8 ffff880429f27fd8 ffff880429f04e70 ffff8800361d9000
[18255.217476] ffff880429f04e70 ffff880429f054e8 0000000000000000 0000000000000000
[18255.226429] Call Trace:
[18255.229788] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.235977] [<ffffffff81457863>] scsi_error_handler+0xa3/0x8b0
[18255.243585] [<ffffffff814577c0>] ? scsi_eh_get_sense+0x250/0x250
[18255.251038] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.257098] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.265032] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.271702] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.279979] scsi_tmf_4 S ffff88042c654d80 0 503 2 0x00000000
[18255.288557] ffff880429f33e08 0000000000000046 ffff880429f05e20 ffff880429f33fd8
[18255.297507] ffff880429f33fd8 ffff880429f33fd8 ffff880429f05e20 ffff880429f05e20
[18255.306464] ffff88042c654d80 ffff88042a433480 ffff88042c654db0 ffff88042c654d80
[18255.315788] Call Trace:
[18255.319523] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.325716] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18255.333058] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18255.339847] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18255.347802] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.353889] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.362169] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.368840] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.376771] scsi_eh_5 S 0000000000000000 0 504 2 0x00000000
[18255.385308] ffff880429f37e18 0000000000000046 ffff880429f06dd0 ffff880429f37fd8
[18255.394606] ffff880429f37fd8 ffff880429f37fd8 ffff880429f06dd0 ffff8800361df000
[18255.403565] ffff880429f06dd0 ffff880429f07448 0000000000000000 0000000000000000
[18255.412529] Call Trace:
[18255.415911] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.422095] [<ffffffff81457863>] scsi_error_handler+0xa3/0x8b0
[18255.429343] [<ffffffff814577c0>] ? scsi_eh_get_sense+0x250/0x250
[18255.436789] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.442862] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.450800] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.457451] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.465359] scsi_tmf_5 S ffff88042c654e80 0 505 2 0x00000000
[18255.474276] ffff880429f43e08 0000000000000046 ffff880429f38000 ffff880429f43fd8
[18255.483246] ffff880429f43fd8 ffff880429f43fd8 ffff880429f38000 ffff880429f38000
[18255.492194] ffff88042c654e80 ffff88042a433a80 ffff88042c654eb0 ffff88042c654e80
[18255.501145] Call Trace:
[18255.504525] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.511070] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18255.518077] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18255.524842] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18255.532776] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.538857] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.547148] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.553946] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.561885] scsi_eh_6 S 0000000000000000 0 506 2 0x00000000
[18255.570431] ffff880429f47e18 0000000000000046 ffff880429f38fb0 ffff880429f47fd8
[18255.579376] ffff880429f47fd8 ffff880429f47fd8 ffff880429f38fb0 ffff8800361d8800
[18255.588682] ffff880429f38fb0 ffff880429f39628 0000000000000000 0000000000000000
[18255.597649] Call Trace:
[18255.601025] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.607214] [<ffffffff81457863>] scsi_error_handler+0xa3/0x8b0
[18255.614486] [<ffffffff814577c0>] ? scsi_eh_get_sense+0x250/0x250
[18255.622305] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.628755] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.636690] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.643339] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.651271] scsi_tmf_6 S ffff88042c654f80 0 507 2 0x00000000
[18255.659925] ffff880429f53e08 0000000000000046 ffff880429f39f60 ffff880429f53fd8
[18255.669286] ffff880429f53fd8 ffff880429f53fd8 ffff880429f39f60 ffff880429f39f60
[18255.678298] ffff88042c654f80 ffff88042a434080 ffff88042c654fb0 ffff88042c654f80
[18255.687261] Call Trace:
[18255.690637] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.696837] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18255.704124] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18255.710899] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18255.718837] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.724931] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.733216] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.740216] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.748136] scsi_eh_7 S 0000000000000000 0 509 2 0x00000000
[18255.756687] ffff880429f83e18 0000000000000046 ffff880429f3bec0 ffff880429f83fd8
[18255.765656] ffff880429f83fd8 ffff880429f83fd8 ffff880429f3bec0 ffff88042bfdf000
[18255.774941] ffff880429f3bec0 ffff880429f3c538 0000000000000000 0000000000000000
[18255.783906] Call Trace:
[18255.787250] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.793443] [<ffffffff81457863>] scsi_error_handler+0xa3/0x8b0
[18255.800703] [<ffffffff814577c0>] ? scsi_eh_get_sense+0x250/0x250
[18255.808137] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.814227] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.822148] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.828814] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.836749] scsi_tmf_7 S ffff88042b859900 0 511 2 0x00000000
[18255.845296] ffff880429f8be08 0000000000000046 ffff880429f3de20 ffff880429f8bfd8
[18255.854261] ffff880429f8bfd8 ffff880429f8bfd8 ffff880429f3de20 ffff880429f3de20
[18255.863224] ffff88042b859900 ffff88042cae0880 ffff88042b859930 ffff88042b859900
[18255.872173] Call Trace:
[18255.875559] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.881749] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18255.888719] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18255.895498] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18255.903424] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18255.909526] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.917452] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18255.924102] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18255.932031] scsi_wq_7 S ffff88042b859a80 0 513 2 0x00000000
[18255.940591] ffff880036337e08 0000000000000046 ffff880036338000 ffff880036337fd8
[18255.949549] ffff880036337fd8 ffff880036337fd8 ffff880036338000 ffff880036338000
[18255.958497] ffff88042b859a80 ffff88042cae1480 ffff88042b859ab0 ffff88042b859a80
[18255.967455] Call Trace:
[18255.970823] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18255.977011] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18255.983985] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18255.990760] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18255.998700] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.004779] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.012700] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.019350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.027274] ttm_swap S ffff880428204180 0 524 2 0x00000000
[18256.035826] ffff88042822be08 0000000000000046 ffff88003633ce70 ffff88042822bfd8
[18256.044786] ffff88042822bfd8 ffff88042822bfd8 ffff88003633ce70 ffff88003633ce70
[18256.053745] ffff880428204180 ffff88042b9bae80 ffff8804282041b0 ffff880428204180
[18256.062716] Call Trace:
[18256.066091] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.072277] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18256.079238] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18256.086019] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18256.093963] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.100059] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.107994] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.114667] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.122611] poll_mpt2sas0_s S ffff88042ba6c480 0 528 2 0x00000000
[18256.131156] ffff880429dffe08 0000000000000046 ffff88003633de20 ffff880429dfffd8
[18256.140121] ffff880429dfffd8 ffff880429dfffd8 ffff88003633de20 ffff88003633de20
[18256.149093] ffff88042ba6c480 ffff88042b9b8880 ffff88042ba6c4b0 ffff88042ba6c480
[18256.158065] Call Trace:
[18256.161443] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.167649] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18256.174625] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18256.181387] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18256.189311] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.195388] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.203308] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.209979] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.217906] kworker/0:1H S ffff88042e016840 0 532 2 0x00000000
[18256.226459] ffff88042bcbfe50 0000000000000046 ffff88042bd00000 ffff88042bcbffd8
[18256.235423] ffff88042bcbffd8 ffff88042bcbffd8 ffff88042bd00000 ffff88042e016860
[18256.244395] ffff88042ba6c6b0 ffff88042bd00000 ffff88042ba6c680 ffff88042e016840
[18256.253339] Call Trace:
[18256.256726] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.262932] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18256.269809] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18256.276970] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.283064] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.290994] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.297663] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.305597] mlx4_en S ffff88042ba6c880 0 541 2 0x00000000
[18256.314141] ffff88041b133e08 0000000000000046 ffff88042bd01f60 ffff88041b133fd8
[18256.323100] ffff88041b133fd8 ffff88041b133fd8 ffff88042bd01f60 ffff88042bd01f60
[18256.332077] ffff88042ba6c880 ffff880036b64280 ffff88042ba6c8b0 ffff88042ba6c880
[18256.341044] Call Trace:
[18256.344422] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.350625] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18256.357604] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18256.364370] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18256.372289] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.378361] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.386278] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.392942] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.400877] kworker/u482:4 S ffff8804ba627400 0 547 2 0x00000000
[18256.409415] ffff88082c8c7e50 0000000000000046 ffff88082c8c9f60 ffff88082c8c7fd8
[18256.418364] ffff88082c8c7fd8 ffff88082c8c7fd8 ffff88082c8c9f60 ffff8804ba627420
[18256.427317] ffff88082c885bb0 ffff88082c8c9f60 ffff88082c885b80 ffff8804ba627400
[18256.436271] Call Trace:
[18256.439649] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.445836] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18256.452700] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18256.459864] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.465953] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.473877] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.480542] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.488459] kworker/14:2 S ffff88082d996480 0 584 2 0x00000000
[18256.497010] ffff880829f3fe50 0000000000000046 ffff880829d3bec0 ffff880829f3ffd8
[18256.505973] ffff880829f3ffd8 ffff880829f3ffd8 ffff880829d3bec0 ffff88082d9964a0
[18256.514937] ffff88082c9b0db0 ffff880829d3bec0 ffff88082c9b0d80 ffff88082d996480
[18256.523882] Call Trace:
[18256.527246] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.533424] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18256.540279] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18256.547426] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.553508] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.561431] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.568081] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.576001] kworker/8:2 S ffff88082d816480 0 585 2 0x00000000
[18256.584542] ffff880829f3be50 0000000000000046 ffff880829d3ce70 ffff880829f3bfd8
[18256.593509] ffff880829f3bfd8 ffff880829f3bfd8 ffff880829d3ce70 ffff88082d8164a0
[18256.602478] ffff88082c30dab0 ffff880829d3ce70 ffff88082c30da80 ffff88082d816480
[18256.611409] Call Trace:
[18256.614774] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.620955] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18256.627814] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18256.634971] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.641048] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.648974] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.655636] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.663552] kworker/14:1H S ffff88082d996840 0 590 2 0x00000000
[18256.672091] ffff88082a7ffe50 0000000000000046 ffff880829d3af10 ffff88082a7fffd8
[18256.681060] ffff88082a7fffd8 ffff88082a7fffd8 ffff880829d3af10 ffff88082d996860
[18256.690023] ffff88082c9b08b0 ffff880829d3af10 ffff88082c9b0880 ffff88082d996840
[18256.698968] Call Trace:
[18256.702327] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.708493] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18256.715346] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18256.722489] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.728565] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.736487] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.743144] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.751076] kworker/13:1H S ffff88082d956840 0 591 2 0x00000000
[18256.759612] ffff880829e7be50 0000000000000046 ffff880829d3de20 ffff880829e7bfd8
[18256.768556] ffff880829e7bfd8 ffff880829e7bfd8 ffff880829d3de20 ffff88082d956860
[18256.777509] ffff88042c749cb0 ffff880829d3de20 ffff88042c749c80 ffff88082d956840
[18256.786436] Call Trace:
[18256.789793] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.795981] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18256.802833] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18256.809987] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.816073] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.824006] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.830665] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.838578] kworker/15:1H S ffff88082d9d6840 0 593 2 0x00000000
[18256.847100] ffff880829d53e50 0000000000000046 ffff880829d3edd0 ffff880829d53fd8
[18256.856041] ffff880829d53fd8 ffff880829d53fd8 ffff880829d3edd0 ffff88082d9d6860
[18256.864991] ffff88082c9d96b0 ffff880829d3edd0 ffff88082c9d9680 ffff88082d9d6840
[18256.873941] Call Trace:
[18256.877293] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.883469] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18256.890318] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18256.897462] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.903535] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.911452] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18256.918105] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.926027] kworker/8:1H S ffff88082d816840 0 595 2 0x00000000
[18256.934566] ffff880829fd3e50 0000000000000046 ffff880829fc0000 ffff880829fd3fd8
[18256.943520] ffff880829fd3fd8 ffff880829fd3fd8 ffff880829fc0000 ffff88082d816860
[18256.952451] ffff88082c30df30 ffff880829fc0000 ffff88082c30df00 ffff88082d816840
[18256.961379] Call Trace:
[18256.964737] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18256.970909] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18256.977764] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18256.984919] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18256.991002] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18256.998922] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.005587] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.013509] kworker/28:1H S ffff88082db16840 0 598 2 0x00000000
[18257.022039] ffff880829ffbe50 0000000000000046 ffff880829fc0fb0 ffff880829ffbfd8
[18257.031001] ffff880829ffbfd8 ffff880829ffbfd8 ffff880829fc0fb0 ffff88082db16860
[18257.039973] ffff88082ca6a7b0 ffff880829fc0fb0 ffff88082ca6a780 ffff88082db16840
[18257.048925] Call Trace:
[18257.052271] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.058442] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.065292] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.072433] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.078510] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.086423] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.093081] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.100993] kworker/25:1H S ffff88082da56840 0 600 2 0x00000000
[18257.109518] ffff880829407e50 0000000000000046 ffff880829fc1f60 ffff880829407fd8
[18257.118474] ffff880829407fd8 ffff880829407fd8 ffff880829fc1f60 ffff88082da56860
[18257.127420] ffff88082c8851b0 ffff880829fc1f60 ffff88082c885180 ffff88082da56840
[18257.136358] Call Trace:
[18257.139714] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.145893] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.152750] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.159892] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.165970] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.173896] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.180549] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.188454] kworker/26:1H S ffff88082da96840 0 607 2 0x00000000
[18257.196973] ffff88082a763e50 0000000000000046 ffff880829fc2f10 ffff88082a763fd8
[18257.205934] ffff88082a763fd8 ffff88082a763fd8 ffff880829fc2f10 ffff88082da96860
[18257.214897] ffff88042bc9f830 ffff880829fc2f10 ffff88042bc9f800 ffff88082da96840
[18257.223849] Call Trace:
[18257.227199] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.233371] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.240220] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.247364] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.253434] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.261350] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.268012] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.275933] kworker/11:1H S ffff88082d8d6840 0 608 2 0x00000000
[18257.284451] ffff880829f0fe50 0000000000000046 ffff880829fc3ec0 ffff880829f0ffd8
[18257.293388] ffff880829f0ffd8 ffff880829f0ffd8 ffff880829fc3ec0 ffff88082d8d6860
[18257.302318] ffff88042c6ef4b0 ffff880829fc3ec0 ffff88042c6ef480 ffff88082d8d6840
[18257.311248] Call Trace:
[18257.314615] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.320806] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.327660] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.334796] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.340868] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.348797] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.355452] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.363363] kworker/12:1H S ffff88082d916840 0 609 2 0x00000000
[18257.371905] ffff880829effe50 0000000000000046 ffff880829fc4e70 ffff880829efffd8
[18257.380852] ffff880829efffd8 ffff880829efffd8 ffff880829fc4e70 ffff88082d916860
[18257.389809] ffff88042c7212b0 ffff880829fc4e70 ffff88042c721280 ffff88082d916840
[18257.398768] Call Trace:
[18257.402125] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.408300] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.415151] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.422297] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.428372] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.436293] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.442951] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.450866] kworker/11:2 S ffff88082d8d6480 0 610 2 0x00000000
[18257.459389] ffff88082a6f3e50 0000000000000046 ffff880829fc5e20 ffff88082a6f3fd8
[18257.468320] ffff88082a6f3fd8 ffff88082a6f3fd8 ffff880829fc5e20 ffff88082d8d64a0
[18257.477279] ffff88042c6ef5b0 ffff880829fc5e20 ffff88042c6ef580 ffff88082d8d6480
[18257.486233] Call Trace:
[18257.489591] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.495757] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.502619] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.509766] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.515844] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.523777] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.530434] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.538352] jbd2/sda1-8 S ffff88082c7c0800 0 613 2 0x00000000
[18257.546893] ffff88041ac53e30 0000000000000046 ffff880829fc6dd0 ffff88041ac53fd8
[18257.555852] ffff88041ac53fd8 ffff88041ac53fd8 ffff880829fc6dd0 ffff88082c7c0828
[18257.564821] ffff88082c7c0be8 ffff88082c7c08a8 ffff88041ac53e80 ffff88082c7c0800
[18257.573776] Call Trace:
[18257.577132] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.583331] [<ffffffffa0268fd3>] kjournald2+0x203/0x260 [jbd2]
[18257.590585] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18257.597750] [<ffffffffa0268dd0>] ? commit_timeout+0x10/0x10 [jbd2]
[18257.605387] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.611467] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.619393] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.626064] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.633984] ext4-rsv-conver S ffff88042c749e80 0 614 2 0x00000000
[18257.642543] ffff88042a60be08 0000000000000046 ffff88041b388000 ffff88042a60bfd8
[18257.651519] ffff88042a60bfd8 ffff88042a60bfd8 ffff88041b388000 ffff88041b388000
[18257.660456] ffff88042c749e80 ffff88082c20c880 ffff88042c749eb0 ffff88042c749e80
[18257.669392] Call Trace:
[18257.672769] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.678964] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18257.685936] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18257.692703] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18257.700636] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.706719] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.714657] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.721304] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.729227] kworker/13:2 S ffff88082d956480 0 616 2 0x00000000
[18257.737792] ffff880829f5be50 0000000000000046 ffff880829f50000 ffff880829f5bfd8
[18257.746771] ffff880829f5bfd8 ffff880829f5bfd8 ffff880829f50000 ffff88082d9564a0
[18257.755736] ffff88042c7492b0 ffff880829f50000 ffff88042c749280 ffff88082d956480
[18257.764693] Call Trace:
[18257.768076] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.774270] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.781139] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.788290] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.794376] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.802306] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.808971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.816903] kworker/27:1H S ffff88082dad6840 0 625 2 0x00000000
[18257.825446] ffff880829fefe50 0000000000000046 ffff880829f50fb0 ffff880829feffd8
[18257.834397] ffff880829feffd8 ffff880829feffd8 ffff880829f50fb0 ffff88082dad6860
[18257.843356] ffff88042bcc7d30 ffff880829f50fb0 ffff88042bcc7d00 ffff88082dad6840
[18257.852320] Call Trace:
[18257.855706] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.861902] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.868768] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.875930] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.882004] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.889941] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.896595] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.904525] kworker/31:1H S ffff88082dbd6840 0 642 2 0x00000000
[18257.913091] ffff880829c17e50 0000000000000046 ffff880829f51f60 ffff880829c17fd8
[18257.922063] ffff880829c17fd8 ffff880829c17fd8 ffff880829f51f60 ffff88082dbd6860
[18257.931021] ffff88042bd41cb0 ffff880829f51f60 ffff88042bd41c80 ffff88082dbd6840
[18257.939982] Call Trace:
[18257.943350] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18257.949521] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18257.956380] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18257.963541] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18257.969628] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.977560] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18257.984221] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18257.992121] kworker/31:2 S ffff88082dbd6480 0 644 2 0x00000000
[18258.000667] ffff88082a747e50 0000000000000046 ffff880829f52f10 ffff88082a747fd8
[18258.009628] ffff88082a747fd8 ffff88082a747fd8 ffff880829f52f10 ffff88082dbd64a0
[18258.018579] ffff88042bd41e30 ffff880829f52f10 ffff88042bd41e00 ffff88082dbd6480
[18258.027529] Call Trace:
[18258.030886] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.037070] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18258.043934] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18258.051085] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.057158] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.065090] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.071748] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.079662] kworker/2:1H S ffff88042e096840 0 664 2 0x00000000
[18258.088192] ffff88041b1e3e50 0000000000000046 ffff88041b38edd0 ffff88041b1e3fd8
[18258.097149] ffff88041b1e3fd8 ffff88041b1e3fd8 ffff88041b38edd0 ffff88042e096860
[18258.106105] ffff88041b2bcfb0 ffff88041b38edd0 ffff88041b2bcf80 ffff88042e096840
[18258.115063] Call Trace:
[18258.118428] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.124603] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18258.131453] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18258.138598] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.144670] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.152601] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.159257] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.167176] kworker/18:1H S ffff88042e296840 0 692 2 0x00000000
[18258.175730] ffff880036aebe50 0000000000000046 ffff88041b38de20 ffff880036aebfd8
[18258.184678] ffff880036aebfd8 ffff880036aebfd8 ffff88041b38de20 ffff88042e296860
[18258.193638] ffff88082ca4a330 ffff88041b38de20 ffff88082ca4a300 ffff88042e296840
[18258.202573] Call Trace:
[18258.205940] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.212124] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18258.218974] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18258.226120] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.232187] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.240109] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.246768] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.254691] kworker/1:1H S ffff88042e056840 0 693 2 0x00000000
[18258.263234] ffff88041b18fe50 0000000000000046 ffff88041b38ce70 ffff88041b18ffd8
[18258.272192] ffff88041b18ffd8 ffff88041b18ffd8 ffff88041b38ce70 ffff88042e056860
[18258.281145] ffff880179222fb0 ffff88041b38ce70 ffff880179222f80 ffff88042e056840
[18258.290086] Call Trace:
[18258.293448] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.299623] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18258.306468] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18258.313615] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.319691] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.327614] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.334280] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.342194] systemd-journal W 0000000000000000 0 700 1 0x00000000
[18258.350731] ffff88042ae4bdb8 0000000000000082 ffff8804287f4e70 ffff88042ae4bfd8
[18258.359685] ffff88042ae4bfd8 ffff88042ae4bfd8 ffff8804287f4e70 0000000000000000
[18258.368638] 0000000000000000 ffff88082a541020 ffff8804287f4e70 0000000000000000
[18258.377580] Call Trace:
[18258.380931] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.387120] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18258.395641] [<ffffffff812484a9>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[18258.403869] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18258.411595] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18258.417875] [<ffffffff811fc928>] ? do_truncate+0x88/0xc0
[18258.424546] [<ffffffff81201268>] ? __sb_start_write+0x58/0x110
[18258.431789] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18258.438636] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18258.445485] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18258.452816] kworker/12:2 S ffff88082d916480 0 703 2 0x00000000
[18258.461345] ffff88082912fe50 0000000000000046 ffff880829f56dd0 ffff88082912ffd8
[18258.470294] ffff88082912ffd8 ffff88082912ffd8 ffff880829f56dd0 ffff88082d9164a0
[18258.479240] ffff88042c7213b0 ffff880829f56dd0 ffff88042c721380 ffff88082d916480
[18258.488195] Call Trace:
[18258.491549] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.497736] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18258.504596] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18258.511737] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.517816] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.525722] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.532366] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.540277] rpciod S ffff88042c6efd80 0 714 2 0x00000000
[18258.548813] ffff88042c713e08 0000000000000046 ffff880829f55e20 ffff88042c713fd8
[18258.557776] ffff88042c713fd8 ffff88042c713fd8 ffff880829f55e20 ffff880829f55e20
[18258.566729] ffff88042c6efd80 ffff88082bda7280 ffff88042c6efdb0 ffff88042c6efd80
[18258.575695] Call Trace:
[18258.579062] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.585239] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18258.592190] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18258.598954] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18258.606871] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.612943] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.620865] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.627507] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.635409] kworker/3:1H S ffff88042e0d6840 0 716 2 0x00000000
[18258.643967] ffff880036a63e50 0000000000000046 ffff88041b388fb0 ffff880036a63fd8
[18258.652937] ffff880036a63fd8 ffff880036a63fd8 ffff88041b388fb0 ffff88042e0d6860
[18258.661908] ffff88042f772d30 ffff88041b388fb0 ffff88042f772d00 ffff88042e0d6840
[18258.670846] Call Trace:
[18258.674206] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.680385] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18258.687235] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18258.694370] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.700437] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.708352] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.715008] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.722930] kworker/26:2 S ffff88082da96480 0 718 2 0x00000000
[18258.731459] ffff88082e6f3e50 0000000000000046 ffff880829f53ec0 ffff88082e6f3fd8
[18258.740408] ffff88082e6f3fd8 ffff88082e6f3fd8 ffff880829f53ec0 ffff88082da964a0
[18258.749362] ffff88042bc9e0b0 ffff880829f53ec0 ffff88042bc9e080 ffff88082da96480
[18258.758314] Call Trace:
[18258.761680] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.767864] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18258.774710] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18258.781856] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.787942] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.795865] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.802520] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.810433] kworker/1:2 S ffff88042e056480 0 723 2 0x00000000
[18258.818974] ffff880036a4fe50 0000000000000046 ffff88041b38bec0 ffff880036a4ffd8
[18258.827947] ffff880036a4ffd8 ffff880036a4ffd8 ffff88041b38bec0 ffff88042e0564a0
[18258.836889] ffff880179223630 ffff88041b38bec0 ffff880179223600 ffff88042e056480
[18258.845846] Call Trace:
[18258.849209] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.855391] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18258.862255] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18258.869407] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18258.875489] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.883416] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18258.890082] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18258.898011] lvmetad S 0000000000000004 0 731 1 0x00000000
[18258.906558] ffff880429bdf8e8 0000000000000086 ffff88042ba03ec0 ffff880429bdffd8
[18258.915505] ffff880429bdffd8 ffff880429bdffd8 ffff88042ba03ec0 0000000000000000
[18258.924443] ffff880429bdfad0 00000000fffffffc 0000000000000004 0000000000000004
[18258.933380] Call Trace:
[18258.936759] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18258.942942] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18258.951457] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18258.958420] [<ffffffff81212f3f>] ? __pollwait+0x7f/0xf0
[18258.964990] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18258.972730] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18258.980175] [<ffffffff81213b91>] do_select+0x6d1/0x7c0
[18258.986631] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18258.994947] [<ffffffff8131e9a1>] ? radix_tree_insert+0x41/0xf0
[18259.002184] [<ffffffff813339ed>] ? list_del+0xd/0x30
[18259.008452] [<ffffffff8118849a>] ? __rmqueue+0x8a/0x460
[18259.015018] [<ffffffff8119b8c0>] ? shmem_add_to_page_cache.isra.19+0xf0/0x170
[18259.023709] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18259.030763] [<ffffffff813339ed>] ? list_del+0xd/0x30
[18259.037026] [<ffffffff8118849a>] ? __rmqueue+0x8a/0x460
[18259.043590] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18259.050641] [<ffffffff8131eb22>] ? radix_tree_lookup_slot+0x22/0x50
[18259.058362] [<ffffffff811806ee>] ? __find_get_page+0x1e/0xa0
[18259.065399] [<ffffffff81183775>] ? filemap_fault+0x215/0x410
[18259.072422] [<ffffffff81213e5b>] core_sys_select+0x1db/0x300
[18259.079442] [<ffffffff811b1431>] ? handle_mm_fault+0x6b1/0xfe0
[18259.086656] [<ffffffff810bd481>] ? enqueue_task+0x51/0x80
[18259.093396] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18259.100613] [<ffffffff8121403a>] SyS_select+0xba/0x110
[18259.107050] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18259.114354] systemd-udevd S 0000000000000000 0 733 1 0x00000000
[18259.122873] ffff88042a47fdb8 0000000000000086 ffff88042ba04e70 ffff88042a47ffd8
[18259.131812] ffff88042a47ffd8 ffff88042a47ffd8 ffff88042ba04e70 0000000000000000
[18259.140744] 0000000000000000 ffff8804b9721da0 ffff88042ba04e70 0000000000000000
[18259.149684] Call Trace:
[18259.153023] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.159170] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18259.167648] [<ffffffff812484a9>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[18259.175844] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18259.183547] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18259.189795] [<ffffffff8120f8e1>] ? do_unlinkat+0xf1/0x2b0
[18259.196515] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18259.203337] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18259.210154] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18259.217459] kworker/29:1H S ffff88082db56840 0 734 2 0x00000000
[18259.225969] ffff880829e3be50 0000000000000046 ffff880829f54e70 ffff880829e3bfd8
[18259.234919] ffff880829e3bfd8 ffff880829e3bfd8 ffff880829f54e70 ffff88082db56860
[18259.243840] ffff88082ca8c7b0 ffff880829f54e70 ffff88082ca8c780 ffff88082db56840
[18259.252742] Call Trace:
[18259.256069] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.262219] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18259.269044] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18259.276149] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18259.282182] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18259.290064] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18259.296683] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18259.304574] kmpathd S ffff88042bc9f780 0 745 2 0x00000000
[18259.313089] ffff88042b687e08 0000000000000046 ffff88041b389f60 ffff88042b687fd8
[18259.322032] ffff88042b687fd8 ffff88042b687fd8 ffff88041b389f60 ffff88041b389f60
[18259.330969] ffff88042bc9f780 ffff88083fa05c80 ffff88042bc9f7b0 ffff88042bc9f780
[18259.339908] Call Trace:
[18259.343240] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.349387] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18259.356313] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18259.363041] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18259.370933] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18259.376982] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18259.384887] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18259.391517] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18259.399415] kmpath_handlerd S ffff88042bc9f000 0 746 2 0x00000000
[18259.407940] ffff88041aaf7e08 0000000000000046 ffff88041b38af10 ffff88041aaf7fd8
[18259.416891] ffff88041aaf7fd8 ffff88041aaf7fd8 ffff88041b38af10 ffff88041b38af10
[18259.425831] ffff88042bc9f000 ffff88083fa05e80 ffff88042bc9f030 ffff88042bc9f000
[18259.434781] Call Trace:
[18259.438144] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.444313] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18259.451265] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18259.458026] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18259.465940] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18259.472022] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18259.479943] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18259.486602] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18259.494525] kworker/15:2 S ffff88082d9d6480 0 751 2 0x00000000
[18259.503065] ffff88082d69fe50 0000000000000046 ffff88082a400000 ffff88082d69ffd8
[18259.512026] ffff88082d69ffd8 ffff88082d69ffd8 ffff88082a400000 ffff88082d9d64a0
[18259.520985] ffff88082c9d9830 ffff88082a400000 ffff88082c9d9800 ffff88082d9d6480
[18259.529941] Call Trace:
[18259.533294] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.539469] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18259.546329] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18259.553478] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18259.559556] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18259.567468] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18259.574130] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18259.582058] multipathd S ffffc900035a7d80 0 767 1 0x00000000
[18259.590610] ffff8800b6ddbcd0 0000000000000086 ffff88042b921f60 ffff8800b6ddbfd8
[18259.599542] ffff8800b6ddbfd8 ffff8800b6ddbfd8 ffff88042b921f60 ffff88042b921f60
[18259.608491] ffff88042b921f60 0000000000000000 ffff8800b6ddbde0 ffffc900035a7d80
[18259.617460] Call Trace:
[18259.620836] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.627031] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18259.634386] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18259.640963] [<ffffffff81183112>] ? __generic_file_aio_write+0x1e2/0x400
[18259.649096] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18259.655373] [<ffffffff8121ef74>] ? mntput+0x24/0x40
[18259.661539] [<ffffffff81200623>] ? __fput+0x183/0x260
[18259.667909] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18259.674269] [<ffffffff8102ab22>] ? do_notify_resume+0x92/0xb0
[18259.681395] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18259.688723] multipathd S ffffc90003729d40 0 770 1 0x00000000
[18259.697240] ffff88042a547cd0 0000000000000086 ffff88042e97af10 ffff88042a547fd8
[18259.706189] ffff88042a547fd8 ffff88042a547fd8 ffff88042e97af10 ffff88042e97af10
[18259.715147] ffff88042e97af10 0000000000000000 ffff88042a547de0 ffffc90003729d40
[18259.724117] Call Trace:
[18259.727484] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.733658] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18259.741001] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18259.747546] [<ffffffff81201268>] ? __sb_start_write+0x58/0x110
[18259.754778] [<ffffffff81229430>] ? __mark_inode_dirty+0xa0/0x290
[18259.762194] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18259.768447] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18259.774791] [<ffffffff8121d245>] ? fd_install+0x25/0x30
[18259.781314] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18259.788634] multipathd S ffff88042c2ffb70 0 832 1 0x00000000
[18259.797144] ffff88042c2ffa00 0000000000000086 ffff88042ba00fb0 ffff88042c2fffd8
[18259.806096] ffff88042c2fffd8 ffff88042c2fffd8 ffff88042ba00fb0 ffff88082bf9f090
[18259.815039] ffff88082bf9f090 ffff88042c2ffb74 ffff88082bf9f000 ffff88042c2ffb70
[18259.823972] Call Trace:
[18259.827307] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.833462] [<ffffffff81689e29>] schedule_timeout+0x239/0x2c0
[18259.840586] [<ffffffff811dc861>] ? __slab_free+0x81/0x2f0
[18259.847312] [<ffffffff811dc861>] ? __slab_free+0x81/0x2f0
[18259.854045] [<ffffffff8156476a>] __skb_recv_datagram+0x3ca/0x460
[18259.861447] [<ffffffff81564380>] ? datagram_poll+0x100/0x100
[18259.868444] [<ffffffff81564841>] skb_recv_datagram+0x41/0x60
[18259.875450] [<ffffffff815a2752>] netlink_recvmsg+0x62/0x450
[18259.882359] [<ffffffff8155615f>] sock_recvmsg+0xbf/0x100
[18259.888972] [<ffffffff810d445d>] ? enqueue_pushable_task+0x6d/0x90
[18259.896561] [<ffffffff81554d9e>] ___sys_recvmsg+0x11e/0x2b0
[18259.903465] [<ffffffff810f8102>] ? do_futex+0x172/0x5b0
[18259.909983] [<ffffffff8168e60b>] ? _raw_spin_unlock_bh+0x1b/0x40
[18259.917373] [<ffffffff8155a3e8>] ? release_sock+0x118/0x170
[18259.924270] [<ffffffff81557131>] __sys_recvmsg+0x51/0x90
[18259.930867] [<ffffffff81557182>] SyS_recvmsg+0x12/0x20
[18259.937273] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18259.944545] multipathd S ffff88042a393b64 0 833 1 0x00000000
[18259.953005] ffff88042a3939f0 0000000000000086 ffff88042ba00000 ffff88042a393fd8
[18259.961911] ffff88042a393fd8 ffff88042a393fd8 ffff88042ba00000 ffff88042a393b48
[18259.970812] 0000000000000000 0000000000000000 0000000000000000 ffff88042a393b64
[18259.979704] Call Trace:
[18259.983019] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18259.989136] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18259.997497] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18260.004504] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18260.013082] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18260.020785] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18260.028197] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18260.034820] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18260.041830] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18260.048834] [<ffffffff8118acea>] ? get_page_from_freelist+0x51a/0x9f0
[18260.056713] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18260.064971] [<ffffffff8118b336>] ? __alloc_pages_nodemask+0x176/0x420
[18260.072845] [<ffffffff811f1107>] ? __mem_cgroup_commit_charge+0xe7/0x370
[18260.081013] [<ffffffff8118ffdb>] ? add_page_to_unevictable_list+0xcb/0x160
[18260.089367] [<ffffffff811dd625>] ? __kmalloc+0x55/0x240
[18260.095884] [<ffffffff810eb0b6>] ? ktime_get_ts64+0x76/0xf0
[18260.102797] [<ffffffff8109abab>] ? recalc_sigpending+0x1b/0x50
[18260.110005] [<ffffffff8109b501>] ? __set_task_blocked+0x41/0xa0
[18260.117296] [<ffffffff81214abe>] SyS_ppoll+0xce/0x1d0
[18260.123627] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18260.130920] multipathd S 00007f4308cd0670 0 839 1 0x00000000
[18260.139399] ffff88042bb9fe68 0000000000000086 ffff88042ba01f60 ffff88042bb9ffd8
[18260.148297] ffff88042bb9ffd8 ffff88042bb9ffd8 ffff88042ba01f60 ffff88042ba01f60
[18260.157207] ffff88042bb9feb8 ffff88042ba01f60 ffff88042bb9ff58 00007f4308cd0670
[18260.166110] Call Trace:
[18260.169416] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18260.175540] [<ffffffff8168b296>] do_nanosleep+0x86/0xf0
[18260.182062] [<ffffffff810b586b>] hrtimer_nanosleep+0xbb/0x180
[18260.189164] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18260.196178] [<ffffffff8168b282>] ? do_nanosleep+0x72/0xf0
[18260.202891] [<ffffffff810b59ae>] SyS_nanosleep+0x7e/0x90
[18260.209503] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18260.216807] multipathd S ffffc9000394d440 0 840 1 0x00000000
[18260.225290] ffff88042bb7fcd0 0000000000000086 ffff88042ba02f10 ffff88042bb7ffd8
[18260.234216] ffff88042bb7ffd8 ffff88042bb7ffd8 ffff88042ba02f10 ffff88042ba02f10
[18260.243150] ffff88042ba02f10 0000000000000000 ffff88042bb7fde0 ffffc9000394d440
[18260.252086] Call Trace:
[18260.255421] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18260.261577] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18260.268900] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18260.275440] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18260.281697] [<ffffffffa02cdd33>] ? dm_ctl_ioctl+0x13/0x20 [dm_mod]
[18260.289316] [<ffffffff81212555>] ? do_vfs_ioctl+0x2d5/0x4b0
[18260.296244] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18260.302571] [<ffffffff812127bb>] ? SyS_ioctl+0x8b/0xc0
[18260.309000] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18260.316308] multipathd S 0000000000004000 0 919 1 0x00000000
[18260.324806] ffff88042b69fc50 0000000000000086 ffff88042e97de20 ffff88042b69ffd8
[18260.333729] ffff88042b69ffd8 ffff88042b69ffd8 ffff88042e97de20 ffff88042a624800
[18260.342655] 0000000000000000 ffff88042a624a10 ffff88042e97de20 0000000000004000
[18260.351553] Call Trace:
[18260.354886] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18260.361036] [<ffffffffa02c7f71>] dm_wait_event+0xa1/0xf0 [dm_mod]
[18260.368535] [<ffffffff81321942>] ? strlcpy+0x42/0x60
[18260.374763] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18260.381888] [<ffffffffa02ce01d>] dev_wait+0x4d/0xd0 [dm_mod]
[18260.388917] [<ffffffff812a8ba8>] ? security_capable+0x18/0x20
[18260.396046] [<ffffffffa02cdfd0>] ? table_deps+0x1a0/0x1a0 [dm_mod]
[18260.403672] [<ffffffffa02cda05>] ctl_ioctl+0x1e5/0x500 [dm_mod]
[18260.410999] [<ffffffffa02cdd33>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[18260.418404] [<ffffffff81212555>] do_vfs_ioctl+0x2d5/0x4b0
[18260.425133] [<ffffffff8109b501>] ? __set_task_blocked+0x41/0xa0
[18260.432438] [<ffffffff812127d1>] SyS_ioctl+0xa1/0xc0
[18260.438679] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18260.445991] multipathd S 0000000000004000 0 941 1 0x00000000
[18260.454494] ffff88042c37fc50 0000000000000086 ffff88042e97ce70 ffff88042c37ffd8
[18260.463420] ffff88042c37ffd8 ffff88042c37ffd8 ffff88042e97ce70 ffff88082bd28000
[18260.472343] 0000000000000000 ffff88082bd28210 ffff88042e97ce70 0000000000004000
[18260.481251] Call Trace:
[18260.484582] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18260.490737] [<ffffffffa02c7f71>] dm_wait_event+0xa1/0xf0 [dm_mod]
[18260.498249] [<ffffffff81321942>] ? strlcpy+0x42/0x60
[18260.504489] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18260.511605] [<ffffffffa02ce01d>] dev_wait+0x4d/0xd0 [dm_mod]
[18260.518633] [<ffffffff812a8ba8>] ? security_capable+0x18/0x20
[18260.525763] [<ffffffffa02cdfd0>] ? table_deps+0x1a0/0x1a0 [dm_mod]
[18260.533373] [<ffffffffa02cda05>] ctl_ioctl+0x1e5/0x500 [dm_mod]
[18260.540695] [<ffffffffa02cdd33>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[18260.548111] [<ffffffff81212555>] do_vfs_ioctl+0x2d5/0x4b0
[18260.554842] [<ffffffff8109b501>] ? __set_task_blocked+0x41/0xa0
[18260.562157] [<ffffffff812127d1>] SyS_ioctl+0xa1/0xc0
[18260.568385] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18260.575688] multipathd S 0000000000004000 0 950 1 0x00000000
[18260.584196] ffff880429d57c50 0000000000000086 ffff88042e978fb0 ffff880429d57fd8
[18260.593124] ffff880429d57fd8 ffff880429d57fd8 ffff88042e978fb0 ffff8800b4779000
[18260.602052] 0000000000000000 ffff8800b4779210 ffff88042e978fb0 0000000000004000
[18260.610968] Call Trace:
[18260.614296] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18260.620451] [<ffffffffa02c7f71>] dm_wait_event+0xa1/0xf0 [dm_mod]
[18260.627971] [<ffffffff81321942>] ? strlcpy+0x42/0x60
[18260.634224] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18260.641347] [<ffffffffa02ce01d>] dev_wait+0x4d/0xd0 [dm_mod]
[18260.648378] [<ffffffff812a8ba8>] ? security_capable+0x18/0x20
[18260.655508] [<ffffffffa02cdfd0>] ? table_deps+0x1a0/0x1a0 [dm_mod]
[18260.663119] [<ffffffffa02cda05>] ctl_ioctl+0x1e5/0x500 [dm_mod]
[18260.670435] [<ffffffffa02cdd33>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[18260.677855] [<ffffffff81212555>] do_vfs_ioctl+0x2d5/0x4b0
[18260.684577] [<ffffffff8109b501>] ? __set_task_blocked+0x41/0xa0
[18260.691881] [<ffffffff812127d1>] SyS_ioctl+0xa1/0xc0
[18260.698120] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18260.705418] multipathd S 0000000000004000 0 960 1 0x00000000
[18260.713928] ffff880036a93c50 0000000000000086 ffff88042e979f60 ffff880036a93fd8
[18260.722865] ffff880036a93fd8 ffff880036a93fd8 ffff88042e979f60 ffff8800b477e000
[18260.731800] 0000000000000000 ffff8800b477e210 ffff88042e979f60 0000000000004000
[18260.740729] Call Trace:
[18260.744074] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18260.750227] [<ffffffffa02c7f71>] dm_wait_event+0xa1/0xf0 [dm_mod]
[18260.757739] [<ffffffff81321942>] ? strlcpy+0x42/0x60
[18260.764002] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18260.771149] [<ffffffffa02ce01d>] dev_wait+0x4d/0xd0 [dm_mod]
[18260.778180] [<ffffffff812a8ba8>] ? security_capable+0x18/0x20
[18260.785310] [<ffffffffa02cdfd0>] ? table_deps+0x1a0/0x1a0 [dm_mod]
[18260.792930] [<ffffffffa02cda05>] ctl_ioctl+0x1e5/0x500 [dm_mod]
[18260.800243] [<ffffffffa02cdd33>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[18260.807653] [<ffffffff81212555>] do_vfs_ioctl+0x2d5/0x4b0
[18260.814385] [<ffffffff8109b501>] ? __set_task_blocked+0x41/0xa0
[18260.821696] [<ffffffff812127d1>] SyS_ioctl+0xa1/0xc0
[18260.827935] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18260.835232] multipathd S 0000000000004000 0 969 1 0x00000000
[18260.843738] ffff880429d2fc50 0000000000000086 ffff88042e978000 ffff880429d2ffd8
[18260.852645] ffff880429d2ffd8 ffff880429d2ffd8 ffff88042e978000 ffff880036a14800
[18260.861551] 0000000000000000 ffff880036a14a10 ffff88042e978000 0000000000004000
[18260.870468] Call Trace:
[18260.873807] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18260.879969] [<ffffffffa02c7f71>] dm_wait_event+0xa1/0xf0 [dm_mod]
[18260.887480] [<ffffffff81321942>] ? strlcpy+0x42/0x60
[18260.893741] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18260.900880] [<ffffffffa02ce01d>] dev_wait+0x4d/0xd0 [dm_mod]
[18260.907910] [<ffffffff812a8ba8>] ? security_capable+0x18/0x20
[18260.915050] [<ffffffffa02cdfd0>] ? table_deps+0x1a0/0x1a0 [dm_mod]
[18260.922685] [<ffffffffa02cda05>] ctl_ioctl+0x1e5/0x500 [dm_mod]
[18260.929997] [<ffffffffa02cdd33>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[18260.937406] [<ffffffff81212555>] do_vfs_ioctl+0x2d5/0x4b0
[18260.944138] [<ffffffff8109b501>] ? __set_task_blocked+0x41/0xa0
[18260.951431] [<ffffffff812127d1>] SyS_ioctl+0xa1/0xc0
[18260.957678] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18260.964987] kworker/9:1H S ffff88082d856840 0 830 2 0x00000000
[18260.973488] ffff880829727e50 0000000000000046 ffff88082a400fb0 ffff880829727fd8
[18260.982416] ffff880829727fd8 ffff880829727fd8 ffff88082a400fb0 ffff88082d856860
[18260.991351] ffff88042c69dab0 ffff88082a400fb0 ffff88042c69da80 ffff88082d856840
[18261.000271] Call Trace:
[18261.003601] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.009746] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18261.016574] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18261.023696] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.029756] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.037675] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.044314] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.052217] kworker/24:1H S ffff88082da16840 0 831 2 0x00000000
[18261.060743] ffff88082906fe50 0000000000000046 ffff88082a401f60 ffff88082906ffd8
[18261.069678] ffff88082906ffd8 ffff88082906ffd8 ffff88082a401f60 ffff88082da16860
[18261.078598] ffff88042bc57830 ffff88082a401f60 ffff88042bc57800 ffff88082da16840
[18261.087530] Call Trace:
[18261.090872] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.097026] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18261.103860] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18261.110991] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.117051] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.124947] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.131584] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.139483] kworker/10:1H S ffff88082d896840 0 881 2 0x00000000
[18261.148023] ffff88082c6bbe50 0000000000000046 ffff88082d610000 ffff88082c6bbfd8
[18261.156963] ffff88082c6bbfd8 ffff88082c6bbfd8 ffff88082d610000 ffff88082d896860
[18261.165913] ffff88042c6c6bb0 ffff88082d610000 ffff88042c6c6b80 ffff88082d896840
[18261.174846] Call Trace:
[18261.178214] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.184378] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18261.191233] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18261.198368] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.204429] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.212331] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.218970] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.226874] kipmi0 S ffff880036b654c8 0 882 2 0x00000000
[18261.235379] ffff88042c72fe48 0000000000000046 ffff88042bd05e20 ffff88042c72ffd8
[18261.244321] ffff88042c72ffd8 ffff88042c72ffd8 ffff88042bd05e20 ffff880036b65400
[18261.253263] ffff880036b65424 0000000000000004 0000000000000246 ffff880036b654c8
[18261.262218] Call Trace:
[18261.265582] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.271756] [<ffffffffa02b1b18>] ipmi_thread+0x118/0x220 [ipmi_si]
[18261.279397] [<ffffffffa02b1a00>] ? poll+0x70/0x70 [ipmi_si]
[18261.286354] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.292434] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.300358] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.307025] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.314950] edac-poller S ffff88042c6c6480 0 883 2 0x00000000
[18261.323486] ffff88042c733e08 0000000000000046 ffff88042bd04e70 ffff88042c733fd8
[18261.332453] ffff88042c733fd8 ffff88042c733fd8 ffff88042bd04e70 ffff88042bd04e70
[18261.341428] ffff88042c6c6480 ffff88082be38e80 ffff88042c6c64b0 ffff88042c6c6480
[18261.350378] Call Trace:
[18261.353750] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.359934] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18261.366903] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18261.373667] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18261.381598] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.387681] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.395622] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.402284] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.410220] kworker/5:2 S ffff88042e156480 0 894 2 0x00000000
[18261.418777] ffff880429dbfe50 0000000000000046 ffff88042bd02f10 ffff880429dbffd8
[18261.427749] ffff880429dbffd8 ffff880429dbffd8 ffff88042bd02f10 ffff88042e1564a0
[18261.436721] ffff88042b6176b0 ffff88042bd02f10 ffff88042b617680 ffff88042e156480
[18261.445677] Call Trace:
[18261.449066] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.455266] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18261.462140] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18261.469297] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.475381] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.483316] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.489985] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.497915] kdmflush S ffff88042c7d2c80 0 912 2 0x00000000
[18261.506448] ffff88042bb77e08 0000000000000046 ffff88042bd06dd0 ffff88042bb77fd8
[18261.515415] ffff88042bb77fd8 ffff88042bb77fd8 ffff88042bd06dd0 ffff88042bd06dd0
[18261.524375] ffff88042c7d2c80 ffff880036b66e80 ffff88042c7d2cb0 ffff88042c7d2c80
[18261.533332] Call Trace:
[18261.536703] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.542892] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18261.549859] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18261.556629] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18261.564558] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.570642] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.578573] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.585236] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.593177] bioset S ffff88042c7d2b00 0 917 2 0x00000000
[18261.601740] ffff8804296f7e08 0000000000000046 ffff88042bacbec0 ffff8804296f7fd8
[18261.610696] ffff8804296f7fd8 ffff8804296f7fd8 ffff88042bacbec0 ffff88042bacbec0
[18261.619641] ffff88042c7d2b00 ffff880036b66080 ffff88042c7d2b30 ffff88042c7d2b00
[18261.628586] Call Trace:
[18261.631964] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.638159] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18261.645119] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18261.651889] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18261.659832] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.665916] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.673839] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.680497] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.688415] kdmwork-253:0 S ffff88042bacedd0 0 918 2 0x00000000
[18261.696954] ffff88042a2a7e68 0000000000000046 ffff88042bacedd0 ffff88042a2a7fd8
[18261.705912] ffff88042a2a7fd8 ffff88042a2a7fd8 ffff88042bacedd0 ffff88042a624bc0
[18261.714872] ffff88042bacedd0 ffff88042bacedd0 0000000000000000 ffff88042bacedd0
[18261.723839] Call Trace:
[18261.727226] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.733412] [<ffffffff810b0e63>] kthread_worker_fn+0x113/0x1d0
[18261.740670] [<ffffffff810b0d50>] ? kthread_stop+0xe0/0xe0
[18261.747429] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.753504] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.761426] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.768094] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.776022] kworker/17:2 S ffff88042e256480 0 933 2 0x00000000
[18261.784575] ffff88042a35be50 0000000000000046 ffff88042bf5bec0 ffff88042a35bfd8
[18261.793534] ffff88042a35bfd8 ffff88042a35bfd8 ffff88042bf5bec0 ffff88042e2564a0
[18261.802485] ffff88082ca283b0 ffff88042bf5bec0 ffff88082ca28380 ffff88042e256480
[18261.811447] Call Trace:
[18261.814819] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.821017] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18261.827893] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18261.835049] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.841139] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.849070] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.855723] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.863651] kdmflush S ffff88042c7d2280 0 937 2 0x00000000
[18261.872217] ffff88041ab1fe08 0000000000000046 ffff88042bf5af10 ffff88041ab1ffd8
[18261.881184] ffff88041ab1ffd8 ffff88041ab1ffd8 ffff88042bf5af10 ffff88042bf5af10
[18261.890155] ffff88042c7d2280 ffff880036b63080 ffff88042c7d22b0 ffff88042c7d2280
[18261.899108] Call Trace:
[18261.902482] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18261.908666] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18261.915623] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18261.922389] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18261.930313] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18261.936398] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.944322] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18261.950991] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18261.958910] bioset S ffff88042c7d2400 0 939 2 0x00000000
[18261.967462] ffff88041b353e08 0000000000000046 ffff88042bf5ce70 ffff88041b353fd8
[18261.976415] ffff88041b353fd8 ffff88041b353fd8 ffff88042bf5ce70 ffff88042bf5ce70
[18261.985364] ffff88042c7d2400 ffff880036b63480 ffff88042c7d2430 ffff88042c7d2400
[18261.994329] Call Trace:
[18261.997694] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.003879] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18262.010851] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18262.017618] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18262.025550] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.031629] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.039547] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.046216] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.054142] kdmwork-253:1 S ffff88042bf5de20 0 940 2 0x00000000
[18262.062696] ffff880429767e68 0000000000000046 ffff88042bf5de20 ffff880429767fd8
[18262.071658] 00000000efd9ed30 ffff880429767fd8 ffff88042bf5de20 ffff88082bd283c0
[18262.080607] ffff88042bf5de20 ffff88042bf5de20 0000000000000000 ffff88042bf5de20
[18262.089557] Call Trace:
[18262.092931] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.099141] [<ffffffff810b0e63>] kthread_worker_fn+0x113/0x1d0
[18262.106397] [<ffffffff810b0d50>] ? kthread_stop+0xe0/0xe0
[18262.113176] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.119283] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.127204] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.133875] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.141809] kworker/22:2 S ffff88042e396480 0 943 2 0x00000000
[18262.150357] ffff88041831be50 0000000000000046 ffff88042bf5edd0 ffff88041831bfd8
[18262.159324] ffff88041831bfd8 ffff88041831bfd8 ffff88042bf5edd0 ffff88042e3964a0
[18262.168297] ffff88042bc057b0 ffff88042bf5edd0 ffff88042bc05780 ffff88042e396480
[18262.177254] Call Trace:
[18262.180630] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.186831] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18262.193700] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18262.200883] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.206983] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.214913] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.221565] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.229493] kdmflush S ffff88042c7d2100 0 944 2 0x00000000
[18262.238068] ffff88042b23be08 0000000000000046 ffff88042ae88000 ffff88042b23bfd8
[18262.247049] ffff88042b23bfd8 ffff88042b23bfd8 ffff88042ae88000 ffff88042ae88000
[18262.255998] ffff88042c7d2100 ffff880036b62a80 ffff88042c7d2130 ffff88042c7d2100
[18262.264973] Call Trace:
[18262.268348] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.274539] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18262.281507] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18262.288282] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18262.296212] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.302297] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.310242] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.316915] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.324844] bioset S ffff88042c7d3b80 0 948 2 0x00000000
[18262.333393] ffff88042b2f7e08 0000000000000046 ffff88042ae88fb0 ffff88042b2f7fd8
[18262.342351] ffff88042b2f7fd8 ffff88042b2f7fd8 ffff88042ae88fb0 ffff88042ae88fb0
[18262.351320] ffff88042c7d3b80 ffff880036b65c80 ffff88042c7d3bb0 ffff88042c7d3b80
[18262.360284] Call Trace:
[18262.363659] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.369858] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18262.376824] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18262.383598] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18262.391527] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.397606] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.405526] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.412189] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.420123] kdmwork-253:2 S ffff88042ae89f60 0 949 2 0x00000000
[18262.428667] ffff88042a46fe68 0000000000000046 ffff88042ae89f60 ffff88042a46ffd8
[18262.437619] ffff88042a46ffd8 ffff88042a46ffd8 ffff88042ae89f60 ffff8800b47793c0
[18262.446570] ffff88042ae89f60 ffff88042ae89f60 0000000000000000 ffff88042ae89f60
[18262.455512] Call Trace:
[18262.458899] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.465101] [<ffffffff810b0e63>] kthread_worker_fn+0x113/0x1d0
[18262.472362] [<ffffffff810b0d50>] ? kthread_stop+0xe0/0xe0
[18262.479139] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.485223] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.493155] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.499818] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.507745] kworker/19:2 S ffff88042e2d6480 0 951 2 0x00000000
[18262.516286] ffff88042a477e50 0000000000000046 ffff88042ae8af10 ffff88042a477fd8
[18262.525250] ffff88042a477fd8 ffff88042a477fd8 ffff88042ae8af10 ffff88042e2d64a0
[18262.534197] ffff88042f03bbb0 ffff88042ae8af10 ffff88042f03bb80 ffff88042e2d6480
[18262.543174] Call Trace:
[18262.546549] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.552737] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18262.559603] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18262.566757] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.572843] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.580778] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.587435] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.595365] kdmflush S ffff88042c7d3d00 0 956 2 0x00000000
[18262.603911] ffff880036a7be08 0000000000000046 ffff88042ae8bec0 ffff880036a7bfd8
[18262.612875] ffff880036a7bfd8 ffff880036a7bfd8 ffff88042ae8bec0 ffff88042ae8bec0
[18262.621833] ffff88042c7d3d00 ffff880036b60080 ffff88042c7d3d30 ffff88042c7d3d00
[18262.630795] Call Trace:
[18262.634164] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.640348] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18262.647312] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18262.654081] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18262.662022] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.668111] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.676042] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.682699] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.690619] bioset S ffff88042c7d3080 0 958 2 0x00000000
[18262.699175] ffff880036103e08 0000000000000046 ffff88042ae8ce70 ffff880036103fd8
[18262.708146] ffff880036103fd8 ffff880036103fd8 ffff88042ae8ce70 ffff88042ae8ce70
[18262.717108] ffff88042c7d3080 ffff880036b67a80 ffff88042c7d30b0 ffff88042c7d3080
[18262.726071] Call Trace:
[18262.729448] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.735635] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18262.742594] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18262.749364] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18262.757309] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.763392] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.771313] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.777967] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.785895] kdmwork-253:3 S ffff88042ae8de20 0 959 2 0x00000000
[18262.794432] ffff880036b4be68 0000000000000046 ffff88042ae8de20 ffff880036b4bfd8
[18262.803390] ffff880036b4bfd8 ffff880036b4bfd8 ffff88042ae8de20 ffff8800b477e3c0
[18262.812340] ffff88042ae8de20 ffff88042ae8de20 0000000000000000 ffff88042ae8de20
[18262.821291] Call Trace:
[18262.824654] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.830845] [<ffffffff810b0e63>] kthread_worker_fn+0x113/0x1d0
[18262.838114] [<ffffffff810b0d50>] ? kthread_stop+0xe0/0xe0
[18262.844881] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.850974] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.858908] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.865557] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.873478] kdmflush S ffff88042c7d2700 0 964 2 0x00000000
[18262.882028] ffff8804182f3e08 0000000000000046 ffff88042ae8edd0 ffff8804182f3fd8
[18262.890999] ffff8804182f3fd8 ffff8804182f3fd8 ffff88042ae8edd0 ffff88042ae8edd0
[18262.899973] ffff88042c7d2700 ffff880036b67e80 ffff88042c7d2730 ffff88042c7d2700
[18262.908947] Call Trace:
[18262.912317] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18262.918508] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18262.925473] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18262.932257] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18262.940198] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18262.946297] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.954227] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18262.960893] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18262.968821] bioset S ffff88042c7d2880 0 967 2 0x00000000
[18262.977363] ffff88042a49be08 0000000000000046 ffff880419670000 ffff88042a49bfd8
[18262.986319] ffff88042a49bfd8 ffff88042a49bfd8 ffff880419670000 ffff880419670000
[18262.995292] ffff88042c7d2880 ffff880036b67480 ffff88042c7d28b0 ffff88042c7d2880
[18263.004263] Call Trace:
[18263.007640] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.013842] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18263.020812] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18263.027574] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18263.035503] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.041587] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.049510] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.056176] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.064101] kdmwork-253:4 S ffff880419670fb0 0 968 2 0x00000000
[18263.072634] ffff88042a4a3e68 0000000000000046 ffff880419670fb0 ffff88042a4a3fd8
[18263.081596] ffff88042a4a3fd8 ffff88042a4a3fd8 ffff880419670fb0 ffff880036a14bc0
[18263.090560] ffff880419670fb0 ffff880419670fb0 0000000000000000 ffff880419670fb0
[18263.099519] Call Trace:
[18263.102906] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.109104] [<ffffffff810b0e63>] kthread_worker_fn+0x113/0x1d0
[18263.116368] [<ffffffff810b0d50>] ? kthread_stop+0xe0/0xe0
[18263.123154] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.129261] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.137203] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.143861] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.151789] kworker/18:2 S ffff88042e296480 0 975 2 0x00000000
[18263.160353] ffff8804182a3e50 0000000000000046 ffff880419671f60 ffff8804182a3fd8
[18263.169304] ffff8804182a3fd8 ffff8804182a3fd8 ffff880419671f60 ffff88042e2964a0
[18263.178270] ffff88082ca4a130 ffff880419671f60 ffff88082ca4a100 ffff88042e296480
[18263.187239] Call Trace:
[18263.190613] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.196814] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18263.203684] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18263.210844] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.216933] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.224870] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.231526] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.239458] kworker/5:1H S ffff88042e156840 0 998 2 0x00000000
[18263.248024] ffff8800b3ce7e50 0000000000000046 ffff8800b3cf0000 ffff8800b3ce7fd8
[18263.256999] ffff8800b3ce7fd8 ffff8800b3ce7fd8 ffff8800b3cf0000 ffff88042e156860
[18263.265964] ffff88042b617d30 ffff8800b3cf0000 ffff88042b617d00 ffff88042e156840
[18263.274927] Call Trace:
[18263.278310] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.284490] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18263.291347] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18263.298501] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.304584] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.312513] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.319189] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.327121] kworker/6:1H S ffff88042e196840 0 1010 2 0x00000000
[18263.335665] ffff88042a217e50 0000000000000046 ffff8804182f9f60 ffff88042a217fd8
[18263.344607] ffff88042a217fd8 ffff88042a217fd8 ffff8804182f9f60 ffff88042e196860
[18263.353545] ffff88042a6d8a30 ffff8804182f9f60 ffff88042a6d8a00 ffff88042e196840
[18263.362506] Call Trace:
[18263.365876] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.372063] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18263.378934] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18263.386082] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.392170] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.400095] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.406763] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.414677] kworker/4:1H S ffff88042e116840 0 1011 2 0x00000000
[18263.423229] ffff880418247e50 0000000000000046 ffff88042b05bec0 ffff880418247fd8
[18263.432183] ffff880418247fd8 ffff880418247fd8 ffff88042b05bec0 ffff88042e116860
[18263.441151] ffff880417e57030 ffff88042b05bec0 ffff880417e57000 ffff88042e116840
[18263.450118] Call Trace:
[18263.453477] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.459651] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18263.466503] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18263.473647] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.479718] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.487634] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.494296] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.502222] kworker/17:1H S ffff88042e256840 0 1014 2 0x00000000
[18263.510770] ffff880417f43e50 0000000000000046 ffff88042b05de20 ffff880417f43fd8
[18263.519735] ffff880417f43fd8 ffff880417f43fd8 ffff88042b05de20 ffff88042e256860
[18263.528677] ffff88082ca29330 ffff88042b05de20 ffff88082ca29300 ffff88042e256840
[18263.537609] Call Trace:
[18263.540965] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.547152] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18263.554007] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18263.561162] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.567242] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.575173] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.581832] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.589757] kworker/7:1H S ffff88042e1d6840 0 1015 2 0x00000000
[18263.598305] ffff88042bf87e50 0000000000000046 ffff88042b05edd0 ffff88042bf87fd8
[18263.607266] ffff88042bf87fd8 ffff88042bf87fd8 ffff88042b05edd0 ffff88042e1d6860
[18263.616228] ffff88042c307fb0 ffff88042b05edd0 ffff88042c307f80 ffff88042e1d6840
[18263.625186] Call Trace:
[18263.628544] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.634719] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18263.641572] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18263.648723] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.654799] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.662723] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.669387] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.677301] kdmflush S ffff88042bd41600 0 1035 2 0x00000000
[18263.685833] ffff88082d207e08 0000000000000046 ffff880829d66dd0 ffff88082d207fd8
[18263.694782] ffff88082d207fd8 ffff88082d207fd8 ffff880829d66dd0 ffff880829d66dd0
[18263.703740] ffff88042bd41600 ffff88042bd77680 ffff88042bd41630 ffff88042bd41600
[18263.712671] Call Trace:
[18263.716035] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.722220] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18263.729182] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18263.735944] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18263.743876] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.749965] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.757891] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.764550] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.772468] kworker/16:1H S ffff88042e216840 0 1043 2 0x00000000
[18263.781008] ffff88041ac13e50 0000000000000046 ffff88042af60000 ffff88041ac13fd8
[18263.789977] ffff88041ac13fd8 ffff88041ac13fd8 ffff88042af60000 ffff88042e216860
[18263.798942] ffff88042c790530 ffff88042af60000 ffff88042c790500 ffff88042e216840
[18263.807894] Call Trace:
[18263.811255] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.817436] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18263.824303] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18263.831460] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.837545] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.845478] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.852135] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.860057] kdmflush S ffff88042bc9f100 0 1050 2 0x00000000
[18263.868588] ffff88082d7cfe08 0000000000000046 ffff88042af60fb0 ffff88082d7cffd8
[18263.877537] ffff88082d7cffd8 ffff88082d7cffd8 ffff88042af60fb0 ffff88042af60fb0
[18263.886493] ffff88042bc9f100 ffff88083fa02280 ffff88042bc9f130 ffff88042bc9f100
[18263.895429] Call Trace:
[18263.898804] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18263.904992] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18263.911952] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18263.918727] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18263.926649] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18263.932732] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.940652] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18263.947311] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18263.955230] bioset S ffff88042bc9e780 0 1052 2 0x00000000
[18263.963786] ffff88082d7dfe08 0000000000000046 ffff88042af61f60 ffff88082d7dffd8
[18263.972735] ffff88082d7dffd8 ffff88082d7dffd8 ffff88042af61f60 ffff88042af61f60
[18263.981669] ffff88042bc9e780 ffff88083fa02680 ffff88042bc9e7b0 ffff88042bc9e780
[18263.990619] Call Trace:
[18263.993982] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.000170] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18264.007138] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18264.013901] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18264.021837] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.027917] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.035845] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.042508] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.050434] kdmflush S ffff88042bc9e700 0 1060 2 0x00000000
[18264.058992] ffff88082d7bbe08 0000000000000046 ffff88042af63ec0 ffff88082d7bbfd8
[18264.067958] ffff88082d7bbfd8 ffff88082d7bbfd8 ffff88042af63ec0 ffff88042af63ec0
[18264.076905] ffff88042bc9e700 ffff88083fa05280 ffff88042bc9e730 ffff88042bc9e700
[18264.085873] Call Trace:
[18264.089249] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.095439] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18264.102397] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18264.109166] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18264.117115] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.123207] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.131142] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.137802] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.145724] bioset S ffff88042bc9f900 0 1062 2 0x00000000
[18264.154277] ffff880823efbe08 0000000000000046 ffff88042af64e70 ffff880823efbfd8
[18264.163263] ffff880823efbfd8 ffff880823efbfd8 ffff88042af64e70 ffff88042af64e70
[18264.172227] ffff88042bc9f900 ffff88083fa05080 ffff88042bc9f930 ffff88042bc9f900
[18264.181194] Call Trace:
[18264.184560] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.190748] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18264.197709] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18264.204472] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18264.212398] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.218475] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.226406] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.233081] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.241008] kdmflush S ffff88042f773080 0 1071 2 0x00000000
[18264.249557] ffff88082a557e08 0000000000000046 ffff88042af66dd0 ffff88082a557fd8
[18264.258514] ffff88082a557fd8 ffff88082a557fd8 ffff88042af66dd0 ffff88042af66dd0
[18264.267483] ffff88042f773080 ffff88042f9f4080 ffff88042f7730b0 ffff88042f773080
[18264.276444] Call Trace:
[18264.279818] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.286018] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18264.292988] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18264.299768] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18264.307690] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.313769] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.321687] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.328358] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.336288] bioset S ffff88042f772d80 0 1073 2 0x00000000
[18264.344836] ffff88082c6f7e08 0000000000000046 ffff88042af65e20 ffff88082c6f7fd8
[18264.353801] ffff88082c6f7fd8 ffff88082c6f7fd8 ffff88042af65e20 ffff88042af65e20
[18264.362776] ffff88042f772d80 ffff88042f9f3a80 ffff88042f772db0 ffff88042f772d80
[18264.371741] Call Trace:
[18264.375117] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.381307] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18264.388283] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18264.395053] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18264.402986] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.409070] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.417004] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.423670] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.431585] kworker/u483:0 S ffff88082a749800 0 1114 2 0x00000000
[18264.440139] ffff88082a3a3e50 0000000000000046 ffff88082bcb0000 ffff88082a3a3fd8
[18264.449122] ffff88082a3a3fd8 ffff88082a3a3fd8 ffff88082bcb0000 ffff88082a749820
[18264.458087] ffff88082c884e30 ffff88082bcb0000 ffff88082c884e00 ffff88082a749800
[18264.467063] Call Trace:
[18264.470441] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.476630] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18264.483494] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18264.490650] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.496729] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.504641] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.511295] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.519237] kworker/u484:0 S ffff88082a749000 0 1115 2 0x00000000
[18264.527779] ffff880036a4be50 0000000000000046 ffff8800b3cbedd0 ffff880036a4bfd8
[18264.536737] ffff880036a4bfd8 ffff880036a4bfd8 ffff8800b3cbedd0 ffff88082a749020
[18264.545690] ffff88082c8847b0 ffff8800b3cbedd0 ffff88082c884780 ffff88082a749000
[18264.554635] Call Trace:
[18264.558005] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.564192] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18264.571064] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18264.578210] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.584292] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.592224] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.598876] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.606788] kworker/u485:0 S ffff88082a748800 0 1116 2 0x00000000
[18264.615350] ffff88083fb43e50 0000000000000046 ffff88082d612f10 ffff88083fb43fd8
[18264.624301] ffff88083fb43fd8 ffff88083fb43fd8 ffff88082d612f10 ffff88082a748820
[18264.633265] ffff88082c885030 ffff88082d612f10 ffff88082c885000 ffff88082a748800
[18264.642229] Call Trace:
[18264.645592] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.651770] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18264.658631] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18264.665790] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.671879] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.679809] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.686463] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.694372] ib-comp-wq S ffff88082c884f80 0 1117 2 0x00000000
[18264.702910] ffff880824b77e08 0000000000000046 ffff88082d616dd0 ffff880824b77fd8
[18264.711867] ffff880824b77fd8 ffff880824b77fd8 ffff88082d616dd0 ffff88082d616dd0
[18264.720822] ffff88082c884f80 ffff88082d52ba80 ffff88082c884fb0 ffff88082c884f80
[18264.729767] Call Trace:
[18264.733126] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.739320] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18264.746278] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18264.753047] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18264.760972] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.767051] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.774974] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.781629] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.789539] ib_addr S ffff88082c884800 0 1118 2 0x00000000
[18264.798091] ffff88082aaf7e08 0000000000000046 ffff88082d615e20 ffff88082aaf7fd8
[18264.807060] ffff88082aaf7fd8 ffff88082aaf7fd8 ffff88082d615e20 ffff88082d615e20
[18264.816001] ffff88082c884800 ffff88082d52c880 ffff88082c884830 ffff88082c884800
[18264.824959] Call Trace:
[18264.828320] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.834502] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18264.841462] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18264.848227] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18264.856153] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.862244] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.870165] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.876827] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.884746] ib_mcast S ffff88082c884c00 0 1119 2 0x00000000
[18264.893283] ffff88081f707e08 0000000000000046 ffff88082d614e70 ffff88081f707fd8
[18264.902247] ffff88081f707fd8 ffff88081f707fd8 ffff88082d614e70 ffff88082d614e70
[18264.911214] ffff88082c884c00 ffff88082d52b680 ffff88082c884c30 ffff88082c884c00
[18264.920184] Call Trace:
[18264.923557] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18264.929743] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18264.936706] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18264.943472] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18264.951399] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18264.957472] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.965394] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18264.972047] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18264.979981] ib_nl_sa_wq S ffff88082c885680 0 1120 2 0x00000000
[18264.988509] ffff8808297ebe08 0000000000000046 ffff88082d611f60 ffff8808297ebfd8
[18264.997465] ffff8808297ebfd8 ffff8808297ebfd8 ffff88082d611f60 ffff88082d611f60
[18265.006432] ffff88082c885680 ffff88082d52ca80 ffff88082c8856b0 ffff88082c885680
[18265.015393] Call Trace:
[18265.018770] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.024963] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.031936] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.038705] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.046634] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.052710] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.060634] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.067302] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.075231] mlx4_ib S ffff88082c884900 0 1121 2 0x00000000
[18265.083762] ffff880824377e08 0000000000000046 ffff88082d613ec0 ffff880824377fd8
[18265.092723] ffff880824377fd8 ffff880824377fd8 ffff88082d613ec0 ffff88082d613ec0
[18265.101664] ffff88082c884900 ffff88082d52b480 ffff88082c884930 ffff88082c884900
[18265.110617] Call Trace:
[18265.113993] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.120198] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.127167] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.133935] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.141880] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.147958] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.155882] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.162549] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.170471] mlx4_ib_mcg S ffff88082c885580 0 1122 2 0x00000000
[18265.179042] ffff880824b6fe08 0000000000000046 ffff88082d610fb0 ffff880824b6ffd8
[18265.188013] ffff880824b6ffd8 ffff880824b6ffd8 ffff88082d610fb0 ffff88082d610fb0
[18265.196975] ffff88082c885580 ffff88082d52ce80 ffff88082c8855b0 ffff88082c885580
[18265.205947] Call Trace:
[18265.209333] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.215516] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.222480] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.229250] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.237178] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.243254] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.251177] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.257853] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.265782] ib_mad1 S ffff880179222080 0 1125 2 0x00000000
[18265.274337] ffff88082bfefe08 0000000000000046 ffff880829d63ec0 ffff88082bfeffd8
[18265.283320] ffff88082bfeffd8 ffff88082bfeffd8 ffff880829d63ec0 ffff880829d63ec0
[18265.292292] ffff880179222080 ffff8800368d1280 ffff8801792220b0 ffff880179222080
[18265.301273] Call Trace:
[18265.304656] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.310848] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.317819] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.324595] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.332533] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.338602] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.346535] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.353211] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.361142] ib_mad2 S ffff880179223f80 0 1126 2 0x00000000
[18265.369672] ffff8804b935fe08 0000000000000046 ffff880829d60000 ffff8804b935ffd8
[18265.378624] ffff8804b935ffd8 ffff8804b935ffd8 ffff880829d60000 ffff880829d60000
[18265.387594] ffff880179223f80 ffff8800b38b5c80 ffff880179223fb0 ffff880179223f80
[18265.396560] Call Trace:
[18265.399930] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.406126] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.413096] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.419872] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.427810] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.433896] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.441832] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.448492] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.456417] iw_cm_wq S ffff88082bdbc900 0 1152 2 0x00000000
[18265.464964] ffff88082e937e08 0000000000000046 ffff880829d61f60 ffff88082e937fd8
[18265.473931] ffff88082e937fd8 ffff88082e937fd8 ffff880829d61f60 ffff880829d61f60
[18265.482884] ffff88082bdbc900 ffff88082d52ae80 ffff88082bdbc930 ffff88082bdbc900
[18265.491856] Call Trace:
[18265.495240] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.501430] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.508403] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.515181] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.523122] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.529205] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.537140] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.543816] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.551746] ib_cm S ffff88082bdbcb00 0 1157 2 0x00000000
[18265.560295] ffff88083fefbe08 0000000000000046 ffff880829d60fb0 ffff88083fefbfd8
[18265.569272] ffff88083fefbfd8 ffff88083fefbfd8 ffff880829d60fb0 ffff880829d60fb0
[18265.578232] ffff88082bdbcb00 ffff88082d52aa80 ffff88082bdbcb30 ffff88082bdbcb00
[18265.587201] Call Trace:
[18265.590583] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.596770] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.603736] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.610506] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.618437] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.624514] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.632440] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.639114] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.647042] rdma_cm S ffff88082bdbcd00 0 1162 2 0x00000000
[18265.655577] ffff88082d22fe08 0000000000000046 ffff880829d62f10 ffff88082d22ffd8
[18265.664537] ffff88082d22ffd8 ffff88082d22ffd8 ffff880829d62f10 ffff880829d62f10
[18265.673491] ffff88082bdbcd00 ffff88082d52d880 ffff88082bdbcd30 ffff88082bdbcd00
[18265.682453] Call Trace:
[18265.685832] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.692030] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.699002] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.705767] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.713685] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.719761] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.727679] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.734352] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.742273] ipoib_flush S ffff88082bdbd000 0 1183 2 0x00000000
[18265.750822] ffff88082ab3fe08 0000000000000046 ffff880829d65e20 ffff88082ab3ffd8
[18265.759782] ffff88082ab3ffd8 ffff88082ab3ffd8 ffff880829d65e20 ffff880829d65e20
[18265.768737] ffff88082bdbd000 ffff88082d52dc80 ffff88082bdbd030 ffff88082bdbd000
[18265.777686] Call Trace:
[18265.781054] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.787254] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.794216] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.800981] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.808905] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.814988] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.822916] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.829579] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.837496] ipoib_wq S ffff88041b24cf80 0 1184 2 0x00000000
[18265.846051] ffff88082d003e08 0000000000000046 ffff88083fde0000 ffff88082d003fd8
[18265.855023] ffff88082d003fd8 ffff88082d003fd8 ffff88083fde0000 ffff88083fde0000
[18265.863989] ffff88041b24cf80 ffff8800b3930880 ffff88041b24cfb0 ffff88041b24cf80
[18265.872954] Call Trace:
[18265.876336] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.882525] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.889487] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.896255] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.904184] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18265.910272] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.918211] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18265.924873] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18265.932794] ipoib_wq S ffff88041b24db80 0 1185 2 0x00000000
[18265.941339] ffff8808247fbe08 0000000000000046 ffff88083fde0fb0 ffff8808247fbfd8
[18265.950312] ffff8808247fbfd8 ffff8808247fbfd8 ffff88083fde0fb0 ffff88083fde0fb0
[18265.959274] ffff88041b24db80 ffff8800b3931a80 ffff88041b24dbb0 ffff88041b24db80
[18265.968249] Call Trace:
[18265.971615] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18265.977811] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18265.984775] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18265.991538] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18265.999470] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18266.005551] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.013476] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18266.020161] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.028085] srp_remove S ffff88082bdbd580 0 1192 2 0x00000000
[18266.036622] ffff8808283c3e08 0000000000000046 ffff88083fde4e70 ffff8808283c3fd8
[18266.045571] ffff8808283c3fd8 ffff8808283c3fd8 ffff88083fde4e70 ffff88083fde4e70
[18266.054544] ffff88082bdbd580 ffff88082d529680 ffff88082bdbd5b0 ffff88082bdbd580
[18266.063505] Call Trace:
[18266.066883] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.073085] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18266.080044] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18266.086811] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18266.094743] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18266.100825] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.108749] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18266.115425] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.123366] target_completi S ffff88082bdbcf00 0 1197 2 0x00000000
[18266.131930] ffff8804b931fe08 0000000000000046 ffff880823a68fb0 ffff8804b931ffd8
[18266.140914] ffff8804b931ffd8 ffff8804b931ffd8 ffff880823a68fb0 ffff880823a68fb0
[18266.149891] ffff88082bdbcf00 ffff88082d528e80 ffff88082bdbcf30 ffff88082bdbcf00
[18266.158856] Call Trace:
[18266.162235] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.168437] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18266.175410] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18266.182194] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18266.190138] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18266.196229] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.204168] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18266.210848] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.218777] tmr-rd_mcp S ffff88082bdbd380 0 1198 2 0x00000000
[18266.227346] ffff880829ddfe08 0000000000000046 ffff880823a69f60 ffff880829ddffd8
[18266.236320] ffff880829ddffd8 ffff880829ddffd8 ffff880823a69f60 ffff880823a69f60
[18266.245291] ffff88082bdbd380 ffff88082d52f280 ffff88082bdbd3b0 ffff88082bdbd380
[18266.254265] Call Trace:
[18266.257636] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.263830] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18266.270801] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18266.277578] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18266.285508] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18266.291585] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.299504] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18266.306174] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.314107] xcopy_wq S ffff88082bdbd300 0 1199 2 0x00000000
[18266.322655] ffff880824273e08 0000000000000046 ffff880823a6af10 ffff880824273fd8
[18266.331621] ffff880824273fd8 ffff880824273fd8 ffff880823a6af10 ffff880823a6af10
[18266.340596] ffff88082bdbd300 ffff88082d528c80 ffff88082bdbd330 ffff88082bdbd300
[18266.349560] Call Trace:
[18266.352938] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.359129] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18266.366103] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18266.372871] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18266.380801] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18266.386873] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.394797] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18266.401462] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.409396] iscsi_eh S ffff88082bdbd800 0 1204 2 0x00000000
[18266.417952] ffff880823eb3e08 0000000000000046 ffff880823a6bec0 ffff880823eb3fd8
[18266.426917] ffff880823eb3fd8 ffff880823eb3fd8 ffff880823a6bec0 ffff880823a6bec0
[18266.435885] ffff88082bdbd800 ffff88082d529080 ffff88082bdbd830 ffff88082bdbd800
[18266.444838] Call Trace:
[18266.448209] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.454411] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18266.461387] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18266.468162] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18266.476097] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18266.482187] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.490111] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18266.496772] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.504691] frwr_recovery S ffff88082bdbd480 0 1213 2 0x00000000
[18266.513243] ffff88082954be08 0000000000000046 ffff880823a6ce70 ffff88082954bfd8
[18266.522232] ffff88082954bfd8 ffff88082954bfd8 ffff880823a6ce70 ffff880823a6ce70
[18266.531199] ffff88082bdbd480 ffff88082d52e280 ffff88082bdbd4b0 ffff88082bdbd480
[18266.540167] Call Trace:
[18266.543535] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.549721] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18266.556683] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18266.563447] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18266.571390] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18266.577462] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.585399] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18266.592057] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.599986] xprtrdma_receiv S ffff88082bdbd180 0 1214 2 0x00000000
[18266.608516] ffff88082954fe08 0000000000000046 ffff880823a6de20 ffff88082954ffd8
[18266.617475] ffff88082954ffd8 ffff88082954ffd8 ffff880823a6de20 ffff880823a6de20
[18266.626435] ffff88082bdbd180 ffff88082d52e080 ffff88082bdbd1b0 ffff88082bdbd180
[18266.635396] Call Trace:
[18266.638759] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.644946] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18266.651917] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18266.658680] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18266.666607] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18266.672682] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.680597] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18266.687264] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18266.695184] auditd S ffff88042a577ea8 0 1241 1 0x00000000
[18266.703703] ffff88042a577db8 0000000000000082 ffff88042ba05e20 ffff88042a577fd8
[18266.712659] ffff88042a577fd8 ffff88042a577fd8 ffff88042ba05e20 ffff88042a577ea8
[18266.721627] 00000000038f9b17 0000000000000000 ffff88042ba05e20 ffff88042a577ea8
[18266.730586] Call Trace:
[18266.733968] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.740162] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18266.748585] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18266.755648] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18266.764270] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18266.772014] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18266.778303] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18266.785172] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18266.792028] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18266.799375] auditd S ffffc9000391b040 0 1244 1 0x00000000
[18266.807915] ffff88041613bcd0 0000000000000082 ffff88042b8a8000 ffff88041613bfd8
[18266.816884] ffff88041613bfd8 ffff88041613bfd8 ffff88042b8a8000 ffff88042b8a8000
[18266.825843] ffff88042b8a8000 0000000000000000 ffff88041613bde0 ffffc9000391b040
[18266.834794] Call Trace:
[18266.838184] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.844381] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18266.851741] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18266.858314] [<ffffffff810cc2dc>] ? set_next_entity+0x3c/0xe0
[18266.865376] [<ffffffff810295ec>] ? __switch_to+0x15c/0x4c0
[18266.872241] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18266.878517] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18266.885662] [<ffffffffa026b802>] ? jbd2_complete_transaction+0x52/0xa0 [jbd2]
[18266.894367] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18266.900729] [<ffffffff81230466>] ? do_fsync+0x76/0xa0
[18266.907093] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18266.914444] irqbalance S 00007ffcaadc6fb0 0 1263 1 0x00000080
[18266.922980] ffff8800b3323e68 0000000000000082 ffff88082c558fb0 ffff8800b3323fd8
[18266.931948] ffff8800b3323fd8 ffff8800b3323fd8 ffff88082c558fb0 ffff88082c558fb0
[18266.940898] ffff8800b3323eb8 ffff88082c558fb0 ffff8800b3323f58 00007ffcaadc6fb0
[18266.949855] Call Trace:
[18266.953210] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18266.959388] [<ffffffff8168b296>] do_nanosleep+0x86/0xf0
[18266.965950] [<ffffffff810b586b>] hrtimer_nanosleep+0xbb/0x180
[18266.973095] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18266.980139] [<ffffffff8168b282>] ? do_nanosleep+0x72/0xf0
[18266.986891] [<ffffffff810b59ae>] SyS_nanosleep+0x7e/0x90
[18266.993536] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.000871] gssproxy S ffff88082d123ea8 0 1264 1 0x00000080
[18267.009415] ffff88082d123db8 0000000000000082 ffff88082bcb4e70 ffff88082d123fd8
[18267.018381] ffff88082d123fd8 ffff88082d123fd8 ffff88082bcb4e70 ffff88082d123ea8
[18267.027347] 0000000001c9c37f 0000000000000000 ffff88082bcb4e70 ffff88082d123ea8
[18267.036304] Call Trace:
[18267.039667] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.045834] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18267.054240] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18267.061277] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18267.069879] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18267.077606] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18267.083874] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18267.090727] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18267.097575] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.104900] gssproxy S ffffc900035bc2c0 0 1266 1 0x00000080
[18267.113426] ffff880429623cd0 0000000000000082 ffff88042d142f10 ffff880429623fd8
[18267.122379] ffff880429623fd8 ffff880429623fd8 ffff88042d142f10 ffff88042d142f10
[18267.131337] ffff88042d142f10 0000000000000000 ffff880429623de0 ffffc900035bc2c0
[18267.140289] Call Trace:
[18267.143653] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.149825] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18267.157158] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18267.163711] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18267.170454] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18267.176701] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18267.183913] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18267.190265] [<ffffffff81692bb5>] ? do_page_fault+0x35/0x90
[18267.197105] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.204424] gssproxy S ffffc900035f9f40 0 1267 1 0x00000080
[18267.212959] ffff88041ac3fcd0 0000000000000082 ffff88042d143ec0 ffff88041ac3ffd8
[18267.221901] ffff88041ac3ffd8 ffff88041ac3ffd8 ffff88042d143ec0 ffff88042d143ec0
[18267.230856] ffff88042d143ec0 0000000000000000 ffff88041ac3fde0 ffffc900035f9f40
[18267.239786] Call Trace:
[18267.243140] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.249329] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18267.256664] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18267.263240] [<ffffffffa036a031>] ? ext4_filemap_fault+0x41/0x50 [ext4]
[18267.271251] [<ffffffff81180b9b>] ? unlock_page+0x2b/0x30
[18267.277911] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18267.284176] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18267.291415] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18267.297761] [<ffffffff81692bb5>] ? do_page_fault+0x35/0x90
[18267.304596] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.311925] gssproxy S ffffc900037a3dc0 0 1268 1 0x00000080
[18267.320455] ffff88042bc7bcd0 0000000000000082 ffff88042d141f60 ffff88042bc7bfd8
[18267.329401] ffff88042bc7bfd8 ffff88042bc7bfd8 ffff88042d141f60 ffff88042d141f60
[18267.338362] ffff88042d141f60 0000000000000000 ffff88042bc7bde0 ffffc900037a3dc0
[18267.347314] Call Trace:
[18267.350674] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.356838] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18267.364172] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18267.370730] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18267.376994] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18267.383340] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.390657] gssproxy S ffffc9000360f3c0 0 1269 1 0x00000080
[18267.399177] ffff88042b907cd0 0000000000000082 ffff88042d140fb0 ffff88042b907fd8
[18267.408130] ffff88042b907fd8 ffff88042b907fd8 ffff88042d140fb0 ffff88042d140fb0
[18267.417077] ffff88042d140fb0 0000000000000000 ffff88042b907de0 ffffc9000360f3c0
[18267.426020] Call Trace:
[18267.429365] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.435531] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18267.442871] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18267.449426] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18267.455665] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18267.462015] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.469327] gssproxy S ffffc9000386fc80 0 1270 1 0x00000080
[18267.477843] ffff88041ac23cd0 0000000000000082 ffff88042d144e70 ffff88041ac23fd8
[18267.486758] ffff88041ac23fd8 ffff88041ac23fd8 ffff88042d144e70 ffff88042d144e70
[18267.495685] ffff88042d144e70 0000000000000000 ffff88041ac23de0 ffffc9000386fc80
[18267.504607] Call Trace:
[18267.507963] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.514137] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18267.521463] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18267.528013] [<ffffffff812011e1>] ? __sb_end_write+0x31/0x60
[18267.534968] [<ffffffff8120768a>] ? pipe_write+0x46a/0x5b0
[18267.541711] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18267.547969] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18267.554317] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.561625] chronyd S 0000000000000007 0 1273 1 0x00000080
[18267.570146] ffff8804161db8e8 0000000000000086 ffff88042b920fb0 ffff8804161dbfd8
[18267.579075] ffff8804161dbfd8 ffff8804161dbfd8 ffff88042b920fb0 ffff8804161dbac8
[18267.587992] 0000000005f5e100 0000000000000000 0000000000000007 0000000000000007
[18267.596916] Call Trace:
[18267.600254] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.606426] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18267.614815] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18267.621864] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18267.630465] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18267.638198] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18267.645631] [<ffffffff81213b91>] do_select+0x6d1/0x7c0
[18267.652098] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18267.660403] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18267.668692] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18267.676977] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18267.685248] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18267.693503] [<ffffffff81621de8>] ? unix_dgram_sendmsg+0x3a8/0x720
[18267.700990] [<ffffffff81555e50>] ? sock_sendmsg+0xb0/0xf0
[18267.707683] [<ffffffff81213e5b>] core_sys_select+0x1db/0x300
[18267.714671] [<ffffffff81556001>] ? SYSC_sendto+0x121/0x1c0
[18267.721455] [<ffffffff810eb08c>] ? ktime_get_ts64+0x4c/0xf0
[18267.728363] [<ffffffff8121403a>] SyS_select+0xba/0x110
[18267.734764] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.742049] polkitd S ffff8804161abbac 0 1274 1 0x00000080
[18267.750522] ffff8804161aba30 0000000000000082 ffff88082c55af10 ffff8804161abfd8
[18267.759428] ffff8804161abfd8 ffff8804161abfd8 ffff88082c55af10 0000000000000000
[18267.768338] ffff8804161abc90 00000000fffffffc 0000000000000000 ffff8804161abbac
[18267.777247] Call Trace:
[18267.780558] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.786684] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18267.795152] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18267.802065] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18267.809761] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18267.817175] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18267.823793] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18267.831185] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18267.838190] [<ffffffff810295ec>] ? __switch_to+0x15c/0x4c0
[18267.844986] [<ffffffff810ce474>] ? update_curr+0x104/0x190
[18267.851777] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18267.858858] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18267.867109] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18267.875359] [<ffffffff810c1f99>] ? ttwu_do_wakeup+0x19/0xd0
[18267.882255] [<ffffffff810c212d>] ? ttwu_do_activate.constprop.91+0x5d/0x70
[18267.890597] [<ffffffff81244fce>] ? inotify_free_event+0xe/0x10
[18267.897775] [<ffffffff811dd173>] ? kfree+0x103/0x140
[18267.904005] [<ffffffff8124bd87>] ? eventfd_ctx_read+0x67/0x210
[18267.911191] [<ffffffff811fe677>] ? vfs_read+0xf7/0x170
[18267.917591] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18267.923790] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18267.931446] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18267.938713] gmain S ffff88041622bbac 0 1275 1 0x00000080
[18267.947184] ffff88041622ba30 0000000000000082 ffff88042b923ec0 ffff88041622bfd8
[18267.956080] ffff88041622bfd8 ffff88041622bfd8 ffff88042b923ec0 0000000000000000
[18267.964961] ffff88041622bc90 00000000fffffffc 0000000000000000 ffff88041622bbac
[18267.973852] Call Trace:
[18267.977152] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18267.983270] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18267.991713] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18267.998604] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18268.006274] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18268.013659] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18268.020277] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18268.027300] [<ffffffff8118acea>] ? get_page_from_freelist+0x51a/0x9f0
[18268.035210] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18268.043498] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18268.051784] [<ffffffffa036a031>] ? ext4_filemap_fault+0x41/0x50 [ext4]
[18268.059772] [<ffffffff81180b9b>] ? unlock_page+0x2b/0x30
[18268.066419] [<ffffffff811acd44>] ? do_read_fault.isra.42+0xe4/0x130
[18268.074132] [<ffffffff8124bd87>] ? eventfd_ctx_read+0x67/0x210
[18268.081352] [<ffffffff811fe677>] ? vfs_read+0xf7/0x170
[18268.087785] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18268.094019] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18268.101704] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18268.109004] gdbus S ffff88082a2e7bac 0 1289 1 0x00000080
[18268.117494] ffff88082a2e7a30 0000000000000082 ffff88082bc13ec0 ffff88082a2e7fd8
[18268.126425] ffff88082a2e7fd8 ffff88082a2e7fd8 ffff88082bc13ec0 0000000000000000
[18268.135343] ffff88082a2e7c90 00000000fffffffc 0000000000000000 ffff88082a2e7bac
[18268.144258] Call Trace:
[18268.147581] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18268.153716] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18268.162199] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18268.169128] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18268.176042] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18268.183735] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18268.191147] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18268.197776] [<ffffffff81622f54>] ? unix_stream_recvmsg+0x54/0x70
[18268.205193] [<ffffffff8161e750>] ? unix_state_double_unlock+0x50/0x50
[18268.213087] [<ffffffff8155615f>] ? sock_recvmsg+0xbf/0x100
[18268.219929] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18268.227052] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18268.235348] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18268.243628] [<ffffffff810c1f99>] ? ttwu_do_wakeup+0x19/0xd0
[18268.250542] [<ffffffff810c212d>] ? ttwu_do_activate.constprop.91+0x5d/0x70
[18268.258932] [<ffffffff810c5308>] ? try_to_wake_up+0x1c8/0x320
[18268.266055] [<ffffffff810c54d2>] ? default_wake_function+0x12/0x20
[18268.273653] [<ffffffff812131a3>] ? pollwake+0x73/0x90
[18268.279999] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18268.286817] [<ffffffff8124bd87>] ? eventfd_ctx_read+0x67/0x210
[18268.294022] [<ffffffff811fe677>] ? vfs_read+0xf7/0x170
[18268.300448] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18268.306670] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18268.314347] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18268.321629] JS GC Helper S ffffc90003683a80 0 1290 1 0x00000080
[18268.330112] ffff88082d287cd0 0000000000000082 ffff88082a63ce70 ffff88082d287fd8
[18268.339039] ffff88082d287fd8 ffff88082d287fd8 ffff88082a63ce70 ffff88082a63ce70
[18268.347936] ffff88082a63ce70 0000000000000000 ffff88082d287de0 ffffc90003683a80
[18268.356818] Call Trace:
[18268.360136] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18268.366277] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18268.373575] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18268.380102] [<ffffffff811bb4cc>] ? page_add_file_rmap+0x8c/0xc0
[18268.387406] [<ffffffff81180b9b>] ? unlock_page+0x2b/0x30
[18268.394020] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18268.400248] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18268.407448] [<ffffffff81321942>] ? strlcpy+0x42/0x60
[18268.413666] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18268.419987] [<ffffffff81692bb5>] ? do_page_fault+0x35/0x90
[18268.426794] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18268.434087] JS Sour~ Thread S ffffc90003674680 0 1291 1 0x00000080
[18268.442575] ffff880823f23cd0 0000000000000082 ffff88082a639f60 ffff880823f23fd8
[18268.451490] ffff880823f23fd8 ffff880823f23fd8 ffff88082a639f60 ffff88082a639f60
[18268.460393] ffff88082a639f60 0000000000000000 ffff880823f23de0 ffffc90003674680
[18268.469307] Call Trace:
[18268.472620] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18268.478740] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18268.486025] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18268.492529] [<ffffffff810c54b0>] ? wake_up_state+0x10/0x20
[18268.499321] [<ffffffff810f54d6>] ? wake_futex+0x66/0x80
[18268.505815] [<ffffffff810f5258>] ? get_futex_key+0x1c8/0x2c0
[18268.512793] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18268.519020] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18268.526201] [<ffffffff81321942>] ? strlcpy+0x42/0x60
[18268.532414] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18268.538707] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18268.545982] runaway-killer- S ffff88082a593ba4 0 1292 1 0x00000080
[18268.554451] ffff88082a593a30 0000000000000082 ffff88082a63bec0 ffff88082a593fd8
[18268.563365] ffff88082a593fd8 ffff88082a593fd8 ffff88082a63bec0 0000000000000000
[18268.572276] ffff88082a593c90 00000000fffffffc 0000000000000000 ffff88082a593ba4
[18268.581197] Call Trace:
[18268.584517] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18268.590657] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18268.599128] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18268.606048] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18268.613745] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18268.621151] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18268.627781] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18268.634803] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18268.641805] [<ffffffff8118acea>] ? get_page_from_freelist+0x51a/0x9f0
[18268.649693] [<ffffffff812003b0>] ? file_free+0x40/0x40
[18268.656131] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18268.663242] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18268.671488] [<ffffffff810c1f55>] ? check_preempt_curr+0x75/0xa0
[18268.678785] [<ffffffff810c1f99>] ? ttwu_do_wakeup+0x19/0xd0
[18268.685689] [<ffffffff810c212d>] ? ttwu_do_activate.constprop.91+0x5d/0x70
[18268.694053] [<ffffffff810c5308>] ? try_to_wake_up+0x1c8/0x320
[18268.701145] [<ffffffff810c54b0>] ? wake_up_state+0x10/0x20
[18268.707944] [<ffffffff810f54d6>] ? wake_futex+0x66/0x80
[18268.714455] [<ffffffff810f5570>] ? futex_wake+0x80/0x160
[18268.721072] [<ffffffff8124bd87>] ? eventfd_ctx_read+0x67/0x210
[18268.728253] [<ffffffff811fe677>] ? vfs_read+0xf7/0x170
[18268.734645] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18268.740853] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18268.748519] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18268.755794] dbus-daemon S 0000000000000000 0 1282 1 0x00000080
[18268.764286] ffff8804162afdb8 0000000000000086 ffff88082c55ce70 ffff8804162affd8
[18268.773182] ffff8804162affd8 ffff8804162affd8 ffff88082c55ce70 0000000000000000
[18268.782102] 0000000000000000 ffff88081e690c60 ffff88082c55ce70 0000000000000000
[18268.791005] Call Trace:
[18268.794326] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18268.800449] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18268.808904] [<ffffffff812484a9>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[18268.817074] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18268.824751] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18268.830984] [<ffffffff81200623>] ? __fput+0x183/0x260
[18268.837304] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18268.844124] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18268.850928] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18268.858224] NetworkManager S ffff88082955bbd4 0 1293 1 0x00000080
[18268.866703] ffff88082955ba30 0000000000000082 ffff88082a60edd0 ffff88082955bfd8
[18268.875626] ffff88082955bfd8 ffff88082955bfd8 ffff88082a60edd0 ffff88082955bb88
[18268.884540] 0000000005f5e100 0000000000000000 0000000000000000 ffff88082955bbd4
[18268.893481] Call Trace:
[18268.896810] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18268.902958] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18268.911335] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18268.918367] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18268.926940] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18268.934640] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18268.942040] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18268.948662] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18268.956069] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18268.963075] [<ffffffff810295ec>] ? __switch_to+0x15c/0x4c0
[18268.969884] [<ffffffff810f5258>] ? get_futex_key+0x1c8/0x2c0
[18268.976890] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18268.984000] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18268.992278] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.000548] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.008806] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.017061] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.025305] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.033534] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.041747] [<ffffffff810eb08c>] ? ktime_get_ts64+0x4c/0xf0
[18269.048606] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18269.054780] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18269.062041] gmain S ffff880823e73bac 0 1302 1 0x00000080
[18269.070481] ffff880823e73a30 0000000000000082 ffff8800b3cbbec0 ffff880823e73fd8
[18269.079374] ffff880823e73fd8 ffff880823e73fd8 ffff8800b3cbbec0 0000000000000000
[18269.088268] ffff880823e73c90 00000000fffffffc 0000000000000000 ffff880823e73bac
[18269.097148] Call Trace:
[18269.100460] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18269.106565] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18269.115023] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18269.121922] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18269.129589] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18269.136971] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18269.143565] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18269.150548] [<ffffffff8118acea>] ? get_page_from_freelist+0x51a/0x9f0
[18269.158417] [<ffffffff811f014d>] ? __mem_cgroup_uncharge_common+0xcd/0x2f0
[18269.166758] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.174998] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.183233] [<ffffffffa036a031>] ? ext4_filemap_fault+0x41/0x50 [ext4]
[18269.191189] [<ffffffff81180b9b>] ? unlock_page+0x2b/0x30
[18269.197776] [<ffffffff811acd44>] ? do_read_fault.isra.42+0xe4/0x130
[18269.205439] [<ffffffff8124bd87>] ? eventfd_ctx_read+0x67/0x210
[18269.212617] [<ffffffff811fe677>] ? vfs_read+0xf7/0x170
[18269.219008] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18269.225194] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18269.232840] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18269.240107] gdbus S ffff88083f93fbb4 0 1307 1 0x00000080
[18269.248557] ffff88083f93fa30 0000000000000082 ffff88082a60de20 ffff88083f93ffd8
[18269.257436] ffff88083f93ffd8 ffff88083f93ffd8 ffff88082a60de20 0000000000000000
[18269.266319] ffff88083f93fc90 00000000fffffffc 0000000000000000 ffff88083f93fbb4
[18269.275190] Call Trace:
[18269.278472] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18269.284573] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18269.293012] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18269.299893] [<ffffffff81212f3f>] ? __pollwait+0x7f/0xf0
[18269.306388] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18269.313266] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18269.320931] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18269.328311] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18269.334903] [<ffffffff81622f54>] ? unix_stream_recvmsg+0x54/0x70
[18269.342274] [<ffffffff8161e750>] ? unix_state_double_unlock+0x50/0x50
[18269.350146] [<ffffffff8155615f>] ? sock_recvmsg+0xbf/0x100
[18269.356948] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18269.364055] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.372346] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.380633] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18269.388909] [<ffffffff810c5308>] ? try_to_wake_up+0x1c8/0x320
[18269.396036] [<ffffffff810c54d2>] ? default_wake_function+0x12/0x20
[18269.403645] [<ffffffff812131a3>] ? pollwake+0x73/0x90
[18269.409990] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18269.416805] [<ffffffff8124bd87>] ? eventfd_ctx_read+0x67/0x210
[18269.424009] [<ffffffff811fe677>] ? vfs_read+0xf7/0x170
[18269.430437] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18269.436655] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18269.444328] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18269.451606] rpc.gssd S 0000000000000000 0 1299 1 0x00000080
[18269.460105] ffff880823ec7db8 0000000000000082 ffff880829cf2f10 ffff880823ec7fd8
[18269.469007] ffff880823ec7fd8 ffff880823ec7fd8 ffff880829cf2f10 0000000000000000
[18269.477907] 0000000000000000[18269.478900] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
Message from[18269.478902] Modules linked in: syslogd@soak-8 rpcsec_gss_krb5at May 9 23:13: nfsv4 dns_resolver41 ...
kernel nfs fscache:NMI watchdog: B osp(OE)UG: soft lockup mdd(OE) lod(OE)- CPU#24 stuck f mdt(OE) lfsck(OE)or 22s! [OI_scru mgs(OE)b:5551]
mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
[18269.478950] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
[18269.478951] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
[18269.478952] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18269.478953] RIP: 0010:[<ffffffffa1217b87>] [<ffffffffa1217b87>] osd_scrub_next+0x167/0x3c0 [osd_ldiskfs]
[18269.478982] RSP: 0018:ffff880703603d00 EFLAGS: 00000282
[18269.478983] RAX: 0000000000000004 RBX: 0000000000000001 RCX: ffff880703603d98
[18269.478983] RDX: ffff880703603d78 RSI: 0000000000000000 RDI: ffff8803162f6000
[18269.478984] RBP: ffff880703603d08 R08: 0000000000000001 R09: 0000000000000004
[18269.478985] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[18269.478985] R13: 0000000000000018 R14: ffff880703603c70 R15: 0000000000000246
[18269.478986] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18269.478987] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18269.478988] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18269.478988] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18269.478989] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18269.478989] Stack:
[18269.478990] 0000000000000001 ffff880703603df0 ffffffffa121cff9 ffffffffa121b990
[18269.478991] ffffffffa1217a20 ffff8800b2a36000 00000000810d354f ffff8803162f6000
[18269.478992] ffff8800b2a37468 0000000020000000 010000000000000c 0000000000000000
[18269.478993] Call Trace:
[18269.478994] [<ffffffffa121cff9>] osd_inode_iteration+0x2a9/0xcc0 [osd_ldiskfs]
[18269.479002] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18269.479008] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18269.479015] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18269.479021] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18269.479023] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18269.479029] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18269.479031] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18269.479033] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18269.479034] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18269.479036] Code: 48 89 df e8 fc 69 47 e0 b8 05 00 00 00 48 8b 75 d0 65 48 33 34 25 28 00 00 00 0f 85 4c 02 00 00 48 83 c4 70 5b 41 5c 41 5d 41 5e <41> 5f 5d c3 48 8b 05 be 58 9c ff 48 85 c0 0f 84 c2 fe ff ff 0f
[18269.919883] ffff88081e6dd620 ffff880829cf2f10 0000000000000000
[18269.928676] Call Trace:
[18269.932363] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18269.938860] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18269.947669] [<ffffffff812484a9>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[18269.956168] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18269.964161] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18269.970689] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18269.977775] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18269.984830] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18269.992352] sssd S ffff8800b330bea8 0 1300 1 0x00000080
[18270.001059] ffff8800b330bdb8 0000000000000082 ffff88042d145e20 ffff8800b330bfd8
[18270.010167] ffff8800b330bfd8 ffff8800b330bfd8 ffff88042d145e20 ffff8800b330bea8
[18270.019234] 0000000001c9c37f 0000000000000000 ffff88042d145e20 ffff8800b330bea8
[18270.028296] Call Trace:
[18270.031752] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.038004] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18270.046492] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18270.053613] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18270.062263] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18270.070036] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18270.076355] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18270.083263] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18270.090172] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.097578] sssd_be S ffff88081e65fea8 0 1301 1300 0x00000080
[18270.106172] ffff88081e65fdb8 0000000000000082 ffff88042c55de20 ffff88081e65ffd8
[18270.115185] ffff88081e65ffd8 ffff88081e65ffd8 ffff88042c55de20 ffff88081e65fea8
[18270.124193] 000000000098967f 0000000000000000 ffff88042c55de20 ffff88081e65fea8
[18270.133195] Call Trace:
[18270.136617] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.142850] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18270.151316] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18270.158426] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18270.167100] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18270.174876] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18270.181191] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18270.188095] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18270.194997] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.202386] sssd_nss S ffff88081e71bea8 0 1304 1300 0x00000080
[18270.210978] ffff88081e71bdb8 0000000000000086 ffff88082bcb5e20 ffff88081e71bfd8
[18270.219980] ffff88081e71bfd8 ffff88081e71bfd8 ffff88082bcb5e20 ffff88081e71bea8
[18270.228957] 000000000098967f 0000000000000000 ffff88082bcb5e20 ffff88081e71bea8
[18270.237947] Call Trace:
[18270.241350] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.247580] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18270.256031] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18270.263111] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18270.271723] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18270.279465] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18270.285743] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18270.292602] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18270.299465] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.306801] sssd_pam S ffff88083f84fea8 0 1305 1300 0x00000080
[18270.315359] ffff88083f84fdb8 0000000000000086 ffff88082bcb6dd0 ffff88083f84ffd8
[18270.324334] ffff88083f84ffd8 ffff88083f84ffd8 ffff88082bcb6dd0 ffff88083f84fea8
[18270.333300] 000000000098967f 0000000000000000 ffff88082bcb6dd0 ffff88083f84fea8
[18270.342270] Call Trace:
[18270.345635] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.351820] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18270.360240] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18270.367301] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18270.375913] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18270.383652] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18270.389930] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18270.396787] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18270.403648] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.410992] sssd_ssh S ffff88083f8bbea8 0 1306 1300 0x00000080
[18270.419543] ffff88083f8bbdb8 0000000000000082 ffff88082bcb2f10 ffff88083f8bbfd8
[18270.428517] ffff88083f8bbfd8 ffff88083f8bbfd8 ffff88082bcb2f10 ffff88083f8bbea8
[18270.437479] 000000000098967f 0000000000000000 ffff88082bcb2f10 ffff88083f8bbea8
[18270.446447] Call Trace:
[18270.449816] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.456014] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18270.464430] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18270.471507] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18270.480133] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18270.487874] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18270.494155] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18270.501017] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18270.507875] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.515220] sssd_pac S ffff88083f917ea8 0 1308 1300 0x00000080
[18270.523777] ffff88083f917db8 0000000000000086 ffff88082bcb1f60 ffff88083f917fd8
[18270.532740] ffff88083f917fd8 ffff88083f917fd8 ffff88082bcb1f60 ffff88083f917ea8
[18270.541705] 000000000098967f 0000000000000000 ffff88082bcb1f60 ffff88083f917ea8
[18270.550676] Call Trace:
[18270.554070] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.560256] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18270.568670] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18270.575725] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18270.584346] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18270.592088] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18270.598375] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18270.605244] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18270.612103] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.619445] systemd-logind S 0000000000000000 0 1315 1 0x00000080
[18270.628016] ffff88083f7d7db8 0000000000000082 ffff88082a63edd0 ffff88083f7d7fd8
[18270.636983] ffff88083f7d7fd8 ffff88083f7d7fd8 ffff88082a63edd0 0000000000000000
[18270.645950] 0000000000000000 ffff8804b9033b60 ffff88082a63edd0 0000000000000000
[18270.654892] Call Trace:
[18270.658271] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.664468] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18270.672978] [<ffffffff812484a9>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
[18270.681215] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18270.688972] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18270.695254] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18270.702124] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18270.708984] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.716332] crond S 0000000000000000 0 1320 1 0x00000080
[18270.724871] ffff88082a4bbe68 0000000000000086 ffff88082a638000 ffff88082a4bbfd8
[18270.733819] ffff88082a4bbfd8 ffff88082a4bbfd8 ffff88082a638000 ffff88082a638000
[18270.742769] ffff88082a4bbeb8 ffff88082a638000 ffff88082a4bbf58 0000000000000000
[18270.751720] Call Trace:
[18270.755091] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.761280] [<ffffffff8168b296>] do_nanosleep+0x86/0xf0
[18270.767844] [<ffffffff810b586b>] hrtimer_nanosleep+0xbb/0x180
[18270.774997] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18270.782044] [<ffffffff8168b282>] ? do_nanosleep+0x72/0xf0
[18270.788797] [<ffffffff810b59ae>] SyS_nanosleep+0x7e/0x90
[18270.795450] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.802782] atd S 0000000000000000 0 1322 1 0x00000080
[18270.811326] ffff88083f97fe68 0000000000000082 ffff88042a68de20 ffff88083f97ffd8
[18270.820276] ffff88083f97ffd8 ffff88083f97ffd8 ffff88042a68de20 ffff88042a68de20
[18270.829218] ffff88083f97feb8 ffff88042a68de20 ffff88083f97ff58 0000000000000000
[18270.838157] Call Trace:
[18270.841513] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.847686] [<ffffffff8168b296>] do_nanosleep+0x86/0xf0
[18270.854245] [<ffffffff810b586b>] hrtimer_nanosleep+0xbb/0x180
[18270.861383] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18270.868424] [<ffffffff8168b282>] ? do_nanosleep+0x72/0xf0
[18270.875157] [<ffffffff810b59ae>] SyS_nanosleep+0x7e/0x90
[18270.881793] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18270.889115] kworker/24:2 S ffff88082da16480 0 1325 2 0x00000080
[18270.897635] ffff88083fc77e50 0000000000000046 ffff880823a6edd0 ffff88083fc77fd8
[18270.906567] ffff88083fc77fd8 ffff88083fc77fd8 ffff880823a6edd0 ffff88082da164a0
[18270.915514] ffff88042bc577b0 ffff880823a6edd0 ffff88042bc57780 ffff88082da16480
[18270.924460] Call Trace:
[18270.927807] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18270.933961] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18270.940795] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18270.947920] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18270.953987] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18270.961892] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18270.968537] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18270.976437] login S ffff88042a687ff0 0 1328 1 0x00000080
[18270.984956] ffff8804b9397e70 0000000000000082 ffff88042a688000 ffff8804b9397fd8
[18270.993890] ffff8804b9397fd8 ffff8804b9397fd8 ffff88042a688000 ffff8804b9397ef8
[18271.002808] ffff88042a6884f0 ffff88042a688000 ffff88042a688000 ffff88042a687ff0
[18271.011724] Call Trace:
[18271.015063] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18271.021221] [<ffffffff8108baf3>] do_wait+0x1f3/0x250
[18271.027461] [<ffffffff8108cbf0>] SyS_wait4+0x80/0x110
[18271.033800] [<ffffffff8108a690>] ? task_stopped_code+0x60/0x60
[18271.041025] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18271.048341] agetty S ffff880416375000 0 1329 1 0x00000080
[18271.056852] ffff88083fb3bcf0 0000000000000082 ffff88042a689f60 ffff88083fb3bfd8
[18271.065777] ffff88083fb3bfd8 ffff88083fb3bfd8 ffff88042a689f60 ffff880417ad6000
[18271.074706] 7fffffffffffffff ffff880416375400 ffff880416375400 ffff880416375000
[18271.083641] Call Trace:
[18271.086986] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18271.093142] [<ffffffff81689e29>] schedule_timeout+0x239/0x2c0
[18271.100276] [<ffffffff810a5bae>] ? get_work_pool+0x3e/0x70
[18271.107111] [<ffffffff810a96db>] ? flush_work+0x15b/0x1c0
[18271.113845] [<ffffffff813e53f8>] ? set_room+0x68/0xc0
[18271.120201] [<ffffffff813e703c>] n_tty_read+0x27c/0x990
[18271.126743] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18271.133576] [<ffffffff813e28ad>] tty_read+0x8d/0x100
[18271.139821] [<ffffffff811fe61e>] vfs_read+0x9e/0x170
[18271.146053] [<ffffffff811ff1ef>] SyS_read+0x7f/0xe0
[18271.152177] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18271.159475] kworker/16:2 S ffff88042e216480 0 1341 2 0x00000080
[18271.167957] ffff8800b2c0fe50 0000000000000046 ffff8800b3cbde20 ffff8800b2c0ffd8
[18271.176868] ffff8800b2c0ffd8 ffff8800b2c0ffd8 ffff8800b3cbde20 ffff88042e2164a0
[18271.185761] ffff88042c790fb0 ffff8800b3cbde20 ffff88042c790f80 ffff88042e216480
[18271.194665] Call Trace:
[18271.198003] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18271.204145] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18271.210970] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18271.218094] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18271.224148] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18271.232044] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18271.238659] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18271.246544] bioset S ffff88041b24c780 0 1381 2 0x00000080
[18271.255057] ffff880823e13e08 0000000000000046 ffff8800b3cb8fb0 ffff880823e13fd8
[18271.263973] ffff880823e13fd8 ffff880823e13fd8 ffff8800b3cb8fb0 ffff8800b3cb8fb0
[18271.272886] ffff88041b24c780 ffff8800b3936680 ffff88041b24c7b0 ffff88041b24c780
[18271.281784] Call Trace:
[18271.285119] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18271.291280] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18271.298210] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18271.304940] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18271.312829] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18271.318873] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18271.326755] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18271.333380] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18271.341270] tuned S 0000000000000000 0 1673 1 0x00000080
[18271.349771] ffff880829cbf8e8 0000000000000086 ffff88083fde3ec0 ffff880829cbffd8
[18271.358692] ffff880829cbffd8 ffff880829cbffd8 ffff88083fde3ec0 ffff880829cbfac8
[18271.367617] 0000000005f5e100 0000000000000000 0000000000000000 0000000000000000
[18271.376548] Call Trace:
[18271.379895] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18271.386067] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18271.394472] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18271.401509] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18271.410105] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18271.417815] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18271.425238] [<ffffffff81213b91>] do_select+0x6d1/0x7c0
[18271.431683] [<ffffffff811a2282>] ? __inc_zone_page_state+0x32/0x60
[18271.439306] [<ffffffff811bb2aa>] ? do_page_add_anon_rmap+0x9a/0xc0
[18271.446918] [<ffffffff811bb2e0>] ? page_add_anon_rmap+0x10/0x20
[18271.454238] [<ffffffff811a2282>] ? __inc_zone_page_state+0x32/0x60
[18271.461855] [<ffffffff811bb2aa>] ? do_page_add_anon_rmap+0x9a/0xc0
[18271.469472] [<ffffffff811bb2e0>] ? page_add_anon_rmap+0x10/0x20
[18271.476792] [<ffffffff811e23f1>] ? remove_migration_pte+0xc1/0x320
[18271.484417] [<ffffffff811e2330>] ? do_pages_stat+0x1a0/0x1a0
[18271.491456] [<ffffffff811bd284>] ? rmap_walk+0x1c4/0x250
[18271.498121] [<ffffffff811f014d>] ? __mem_cgroup_uncharge_common+0xcd/0x2f0
[18271.506539] [<ffffffff81195063>] ? putback_lru_page+0x53/0xb0
[18271.513678] [<ffffffff811e41fb>] ? migrate_pages+0x43b/0x790
[18271.520710] [<ffffffff810ccedb>] ? should_numa_migrate_memory+0x5b/0x150
[18271.528911] [<ffffffff81213e5b>] core_sys_select+0x1db/0x300
[18271.535952] [<ffffffff811b1457>] ? handle_mm_fault+0x6d7/0xfe0
[18271.543190] [<ffffffff810eb08c>] ? ktime_get_ts64+0x4c/0xf0
[18271.550126] [<ffffffff810eb08c>] ? ktime_get_ts64+0x4c/0xf0
[18271.557052] [<ffffffff8121403a>] SyS_select+0xba/0x110
[18271.563492] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18271.570806] gmain S ffff880826e3fba4 0 1940 1 0x00000080
[18271.579336] ffff880826e3fa30 0000000000000086 ffff880825ed0fb0 ffff880826e3ffd8
[18271.588272] ffff880826e3ffd8 ffff880826e3ffd8 ffff880825ed0fb0 0000000000000000
[18271.597198] ffff880826e3fc90 00000000fffffffc 0000000000000000 ffff880826e3fba4
[18271.606128] Call Trace:
[18271.609474] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18271.615618] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18271.624119] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18271.631062] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18271.638772] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18271.646200] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18271.652843] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18271.659874] [<ffffffff811a2599>] ? zone_statistics+0x89/0xa0
[18271.666892] [<ffffffff8118acea>] ? get_page_from_freelist+0x51a/0x9f0
[18271.674782] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18271.683066] [<ffffffff81183775>] ? filemap_fault+0x215/0x410
[18271.690106] [<ffffffffa036a031>] ? ext4_filemap_fault+0x41/0x50 [ext4]
[18271.698093] [<ffffffff81180b9b>] ? unlock_page+0x2b/0x30
[18271.704726] [<ffffffff811acd44>] ? do_read_fault.isra.42+0xe4/0x130
[18271.712436] [<ffffffff8124bd87>] ? eventfd_ctx_read+0x67/0x210
[18271.719633] [<ffffffff811fe677>] ? vfs_read+0xf7/0x170
[18271.726060] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18271.732285] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18271.739959] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18271.747252] tuned S ffff88082bddfbac 0 1948 1 0x00000080
[18271.755733] ffff88082bddfa30 0000000000000086 ffff880825ed1f60 ffff88082bddffd8
[18271.764645] ffff88082bddffd8 ffff88082bddffd8 ffff880825ed1f60 0000000000000000
[18271.773561] ffff88082bddfc90 00000000fffffffc 0000000000000000 ffff88082bddfbac
[18271.782504] Call Trace:
[18271.785831] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18271.791970] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18271.800454] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18271.807387] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18271.814308] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18271.822008] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18271.829415] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18271.836060] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18271.843471] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18271.850501] [<ffffffff810295ec>] ? __switch_to+0x15c/0x4c0
[18271.857322] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18271.864440] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18271.872717] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18271.880985] [<ffffffff810c1f99>] ? ttwu_do_wakeup+0x19/0xd0
[18271.887893] [<ffffffff810c212d>] ? ttwu_do_activate.constprop.91+0x5d/0x70
[18271.896261] [<ffffffff810c5308>] ? try_to_wake_up+0x1c8/0x320
[18271.903383] [<ffffffff810c54b0>] ? wake_up_state+0x10/0x20
[18271.910210] [<ffffffff810f54d6>] ? wake_futex+0x66/0x80
[18271.916742] [<ffffffff810f5570>] ? futex_wake+0x80/0x160
[18271.923356] [<ffffffff810f80b2>] ? do_futex+0x122/0x5b0
[18271.929855] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18271.936651] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18271.942857] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18271.950525] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18271.957796] tuned S 0000000000000000 0 1952 1 0x00000080
[18271.966281] ffff88082475f8e8 0000000000000086 ffff880825ed2f10 ffff88082475ffd8
[18271.975181] ffff88082475ffd8 ffff88082475ffd8 ffff880825ed2f10 ffff88082475fac8
[18271.984070] 00000000000f423f 0000000000000000 0000000000000000 0000000000000000
[18271.992951] Call Trace:
[18271.996267] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18272.002390] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18272.010751] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18272.017761] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18272.026324] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18272.034025] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18272.041422] [<ffffffff81213b91>] do_select+0x6d1/0x7c0
[18272.047846] [<ffffffff813339ed>] ? list_del+0xd/0x30
[18272.054081] [<ffffffff811a2282>] ? __inc_zone_page_state+0x32/0x60
[18272.061665] [<ffffffff811bb2aa>] ? do_page_add_anon_rmap+0x9a/0xc0
[18272.069249] [<ffffffff811bb2e0>] ? page_add_anon_rmap+0x10/0x20
[18272.076552] [<ffffffff811e23f1>] ? remove_migration_pte+0xc1/0x320
[18272.084150] [<ffffffff811e2330>] ? do_pages_stat+0x1a0/0x1a0
[18272.091153] [<ffffffff811bd284>] ? rmap_walk+0x1c4/0x250
[18272.097759] [<ffffffff811f014d>] ? __mem_cgroup_uncharge_common+0xcd/0x2f0
[18272.106111] [<ffffffff81195063>] ? putback_lru_page+0x53/0xb0
[18272.113216] [<ffffffff811e41fb>] ? migrate_pages+0x43b/0x790
[18272.120212] [<ffffffff81213e5b>] core_sys_select+0x1db/0x300
[18272.127198] [<ffffffff811b1457>] ? handle_mm_fault+0x6d7/0xfe0
[18272.134378] [<ffffffff810eb08c>] ? ktime_get_ts64+0x4c/0xf0
[18272.141251] [<ffffffff810eb08c>] ? ktime_get_ts64+0x4c/0xf0
[18272.148115] [<ffffffff8121403a>] SyS_select+0xba/0x110
[18272.154508] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18272.161766] tuned S 0000000000000000 0 1963 1 0x00000080
[18272.170242] ffff880829c23db8 0000000000000086 ffff88082d54de20 ffff880829c23fd8
[18272.179129] ffff880829c23fd8 ffff880829c23fd8 ffff88082d54de20 0000000000000000
[18272.188011] 0000000000000000 ffff88082bfa33e0 ffff88082d54de20 0000000000000000
[18272.196878] Call Trace:
[18272.200180] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18272.206302] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18272.214749] [<ffffffff810c54b0>] ? wake_up_state+0x10/0x20
[18272.221559] [<ffffffff810f54d6>] ? wake_futex+0x66/0x80
[18272.228071] [<ffffffff810f5570>] ? futex_wake+0x80/0x160
[18272.234673] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18272.242342] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18272.248562] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18272.255735] [<ffffffff81248fb8>] ? SyS_epoll_ctl+0x118/0xa20
[18272.262730] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18272.269534] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18272.276340] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18272.283634] rsyslogd S 0000000000000000 0 1678 1 0x00000080
[18272.292145] ffff88082e5f38e8 0000000000000082 ffff880823a68000 ffff88082e5f3fd8
[18272.301051] ffff88082e5f3fd8 ffff88082e5f3fd8 ffff880823a68000 ffff88082e5f3ac8
[18272.309940] 0000000005f5e100 0000000000000000 0000000000000000 0000000000000000
[18272.318841] Call Trace:
[18272.322168] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18272.328294] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18272.336652] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18272.343664] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18272.352224] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18272.359908] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18272.367298] [<ffffffff81213b91>] do_select+0x6d1/0x7c0
[18272.373713] [<ffffffff8124894e>] ? ep_poll_callback+0xee/0x210
[18272.380904] [<ffffffff810bc4af>] ? __wake_up_sync_key+0x4f/0x60
[18272.388197] [<ffffffff815584da>] ? sock_def_readable+0x3a/0x70
[18272.395390] [<ffffffff81621de8>] ? unix_dgram_sendmsg+0x3a8/0x720
[18272.402876] [<ffffffff81555e50>] ? sock_sendmsg+0xb0/0xf0
[18272.409589] [<ffffffff810295ec>] ? __switch_to+0x15c/0x4c0
[18272.416391] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18272.423483] [<ffffffff81213e5b>] core_sys_select+0x1db/0x300
[18272.430483] [<ffffffff81214f05>] ? __d_free+0x35/0x40
[18272.436792] [<ffffffff81215400>] ? d_free+0x60/0x70
[18272.442904] [<ffffffff81215ace>] ? dentry_kill+0x14e/0x1b0
[18272.449693] [<ffffffff8121ee6e>] ? mntput_no_expire+0x3e/0x120
[18272.456873] [<ffffffff8121ef74>] ? mntput+0x24/0x40
[18272.462981] [<ffffffff81200623>] ? __fput+0x183/0x260
[18272.469290] [<ffffffff810eb08c>] ? ktime_get_ts64+0x4c/0xf0
[18272.476175] [<ffffffff8121403a>] SyS_select+0xba/0x110
[18272.482588] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18272.489869] in:imjournal S ffff88082776fba4 0 1689 1 0x00000080
[18272.498367] ffff88082776fa30 0000000000000082 ffff88082c783ec0 ffff88082776ffd8
[18272.507290] ffff88082776ffd8 ffff88082776ffd8 ffff88082c783ec0 0000000000000000
[18272.516193] ffff88082776fc90 00000000fffffffc 0000000000000000 ffff88082776fba4
[18272.525086] Call Trace:
[18272.528408] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18272.534535] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18272.543007] [<ffffffffa03a0e7e>] ? ext4_free_blocks+0x67e/0xb80 [ext4]
[18272.550977] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18272.557884] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18272.565574] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18272.572975] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18272.579604] [<ffffffffa0269130>] ? jbd2_journal_release_jbd_inode+0x100/0x130 [jbd2]
[18272.588952] [<ffffffff810bc454>] ? __wake_up+0x44/0x50
[18272.595399] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18272.602525] [<ffffffff810d060c>] ? enqueue_entity+0x26c/0xb60
[18272.609645] [<ffffffff810d1108>] ? enqueue_task_fair+0x208/0x6c0
[18272.617044] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18272.625338] [<ffffffff810c212d>] ? ttwu_do_activate.constprop.91+0x5d/0x70
[18272.633710] [<ffffffff810c5308>] ? try_to_wake_up+0x1c8/0x320
[18272.640816] [<ffffffff811dc861>] ? __slab_free+0x81/0x2f0
[18272.647541] [<ffffffff81215ace>] ? dentry_kill+0x14e/0x1b0
[18272.654356] [<ffffffff810c54b0>] ? wake_up_state+0x10/0x20
[18272.661158] [<ffffffff810f54d6>] ? wake_futex+0x66/0x80
[18272.667670] [<ffffffff810f5982>] ? futex_wake_op+0x332/0x570
[18272.674661] [<ffffffff810f8102>] ? do_futex+0x172/0x5b0
[18272.681171] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18272.688250] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18272.694466] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18272.702131] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18272.709398] rs:main Q:Reg S ffffc900037c3900 0 1690 1 0x00000080
[18272.717864] ffff8808267fbcd0 0000000000000082 ffff88082c784e70 ffff8808267fbfd8
[18272.726766] ffff8808267fbfd8 ffff8808267fbfd8 ffff88082c784e70 ffff88082c784e70
[18272.735672] ffff88082c784e70 0000000000000000 ffff8808267fbde0 ffffc900037c3900
[18272.744578] Call Trace:
[18272.747890] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18272.754022] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18272.761331] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18272.767858] [<ffffffffa035cf0b>] ? ext4_file_write+0xdb/0x470 [ext4]
[18272.775646] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18272.781883] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18272.788217] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18272.795531] munged S ffff88042c2d7e30 0 1681 1 0x00000080
[18272.804051] ffff88042c2d7cc0 0000000000000082 ffff88042bae2f10 ffff88042c2d7fd8
[18272.812982] ffff88042c2d7fd8 ffff88042c2d7fd8 ffff88042bae2f10 ffff880414170490
[18272.821908] ffff880414170490 ffff88042c2d7e34 ffff880414170400 ffff88042c2d7e30
[18272.830842] Call Trace:
[18272.834207] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18272.840381] [<ffffffff81689e29>] schedule_timeout+0x239/0x2c0
[18272.847518] [<ffffffff8156476a>] __skb_recv_datagram+0x3ca/0x460
[18272.854934] [<ffffffff81564380>] ? datagram_poll+0x100/0x100
[18272.861957] [<ffffffff81564841>] skb_recv_datagram+0x41/0x60
[18272.868972] [<ffffffff8161ee37>] unix_accept+0x97/0x170
[18272.875509] [<ffffffff81555743>] SYSC_accept4+0x103/0x220
[18272.882227] [<ffffffff812011e1>] ? __sb_end_write+0x31/0x60
[18272.889132] [<ffffffff811fe862>] ? vfs_write+0x172/0x1e0
[18272.895739] [<ffffffff81556b3e>] SyS_accept4+0xe/0x10
[18272.902067] [<ffffffff81556b50>] SyS_accept+0x10/0x20
[18272.908381] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18272.915661] munged S ffffc90003652b40 0 1683 1 0x00000080
[18272.924161] ffff88042d647cd0 0000000000000082 ffff88042e97bec0 ffff88042d647fd8
[18272.933051] ffff88042d647fd8 ffff88042d647fd8 ffff88042e97bec0 ffff88042e97bec0
[18272.941941] ffff88042e97bec0 ffff88042d647d60 ffff88042d647de0 ffffc90003652b40
[18272.950832] Call Trace:
[18272.954157] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18272.960285] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18272.967591] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18272.974099] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18272.981103] [<ffffffff810f53f2>] ? futex_wait_queue_me+0xa2/0x120
[18272.988601] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18272.994824] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18273.001154] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18273.008456] munged S ffffc900035b9440 0 1684 1 0x00000080
[18273.016963] ffff88041929fcd0 0000000000000082 ffff88042d140000 ffff88041929ffd8
[18273.025886] ffff88041929ffd8 ffff88041929ffd8 ffff88042d140000 ffff88042d140000
[18273.034813] ffff88042d140000 0000000000000000 ffff88041929fde0 ffffc900035b9440
[18273.043740] Call Trace:
[18273.047078] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18273.053228] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18273.060540] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18273.067067] [<ffffffff810ce95c>] ? dequeue_entity+0x11c/0x5d0
[18273.074182] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18273.081585] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18273.087797] [<ffffffff8168c3c9>] ? schedule+0x29/0x70
[18273.094107] [<ffffffff8109abab>] ? recalc_sigpending+0x1b/0x50
[18273.101302] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18273.107616] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18273.114898] munged S ffffc900035b9440 0 1685 1 0x00000080
[18273.123394] ffff880419fe3cd0 0000000000000082 ffff88042d146dd0 ffff880419fe3fd8
[18273.132318] ffff880419fe3fd8 ffff880419fe3fd8 ffff88042d146dd0 ffff88042d146dd0
[18273.141223] ffff88042d146dd0 0000000000000000 ffff880419fe3de0 ffffc900035b9440
[18273.150132] Call Trace:
[18273.153459] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18273.159605] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18273.166898] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18273.173409] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18273.179622] [<ffffffff8109abab>] ? recalc_sigpending+0x1b/0x50
[18273.186815] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18273.193134] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18273.200431] xinetd S ffff880825703bb4 0 1687 1 0x00000080
[18273.208930] ffff880825703a30 0000000000000082 ffff88082c406dd0 ffff880825703fd8
[18273.217844] ffff880825703fd8 ffff880825703fd8 ffff88082c406dd0 0000000000000000
[18273.226766] ffff880825703c90 00000000fffffffc 0000000000000000 ffff880825703bb4
[18273.235691] Call Trace:
[18273.239040] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18273.245206] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18273.253709] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18273.260668] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18273.267593] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18273.275292] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18273.282709] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18273.289345] [<ffffffff810ba628>] ? __wake_up_common+0x58/0x90
[18273.296468] [<ffffffff810bc4af>] ? __wake_up_sync_key+0x4f/0x60
[18273.303772] [<ffffffff815584da>] ? sock_def_readable+0x3a/0x70
[18273.310974] [<ffffffff81621de8>] ? unix_dgram_sendmsg+0x3a8/0x720
[18273.318495] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18273.326779] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18273.335062] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18273.343339] [<ffffffff81180b9b>] ? unlock_page+0x2b/0x30
[18273.349956] [<ffffffff811acd44>] ? do_read_fault.isra.42+0xe4/0x130
[18273.357639] [<ffffffff811b1431>] ? handle_mm_fault+0x6b1/0xfe0
[18273.364818] [<ffffffff81556001>] ? SYSC_sendto+0x121/0x1c0
[18273.371616] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18273.378804] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18273.385007] [<ffffffff8168edc8>] ? page_fault+0x28/0x30
[18273.391501] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18273.398772] sshd S 0000000000000005 0 1692 1 0x00000080
[18273.407266] ffff88042b71b8e8 0000000000000086 ffff88042cb00000 ffff88042b71bfd8
[18273.416168] ffff88042b71bfd8 ffff88042b71bfd8 ffff88042cb00000 0000000000000000
[18273.425065] ffff88042b71bad0 00000000fffffffc 0000000000000005 0000000000000005
[18273.433947] Call Trace:
[18273.437276] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18273.443411] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18273.451879] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18273.458785] [<ffffffff81212f3f>] ? __pollwait+0x7f/0xf0
[18273.465312] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18273.472985] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18273.480371] [<ffffffff81213b91>] do_select+0x6d1/0x7c0
[18273.486787] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18273.495045] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18273.503305] [<ffffffff81321ec2>] ? put_dec+0x72/0x90
[18273.509512] [<ffffffff81322eb3>] ? number.isra.2+0x323/0x360
[18273.516503] [<ffffffff812431b7>] ? fsnotify_clear_marks_by_inode+0xa7/0x140
[18273.524949] [<ffffffff81325191>] ? vsnprintf+0x441/0x6a0
[18273.531567] [<ffffffff8121a6a8>] ? destroy_inode+0x38/0x60
[18273.538373] [<ffffffff8140e456>] ? mix_pool_bytes.constprop.18+0x56/0xa0
[18273.546534] [<ffffffff81213e5b>] core_sys_select+0x1db/0x300
[18273.553525] [<ffffffff8108ae25>] ? wait_consider_task+0xa5/0xb80
[18273.560898] [<ffffffff81036629>] ? __restore_xstate_sig+0x99/0x5d0
[18273.568471] [<ffffffff8121403a>] SyS_select+0xba/0x110
[18273.574868] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18273.582137] automount S 0000000000000000 0 1694 1 0x00000080
[18273.590585] ffff880825a2fd90 0000000000000082 ffff88082d6f4e70 ffff880825a2ffd8
[18273.599473] ffff880825a2ffd8 ffff880825a2ffd8 ffff88082d6f4e70 ffff88082d6f4e70
[18273.608350] 7fffffffffffffff ffff880825a2fed0 ffff88082d6f4e70 0000000000000000
[18273.617233] Call Trace:
[18273.620530] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18273.626640] [<ffffffff81689e29>] schedule_timeout+0x239/0x2c0
[18273.633725] [<ffffffff81180b9b>] ? unlock_page+0x2b/0x30
[18273.640333] [<ffffffff811acd44>] ? do_read_fault.isra.42+0xe4/0x130
[18273.647997] [<ffffffff8109abab>] ? recalc_sigpending+0x1b/0x50
[18273.655186] [<ffffffff81689ece>] schedule_timeout_interruptible+0x1e/0x20
[18273.663452] [<ffffffff8109e982>] do_sigtimedwait+0x1c2/0x2a0
[18273.670457] [<ffffffff8109eae5>] SYSC_rt_sigtimedwait+0x85/0xf0
[18273.677755] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18273.684962] [<ffffffff81692bb5>] ? do_page_fault+0x35/0x90
[18273.691780] [<ffffffff8109eb5e>] SyS_rt_sigtimedwait+0xe/0x10
[18273.698886] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18273.706193] automount S ffffc90003932180 0 1698 1 0x00000080
[18273.714692] ffff88042d107cd0 0000000000000082 ffff88042b924e70 ffff88042d107fd8
[18273.723637] ffff88042d107fd8 ffff88042d107fd8 ffff88042b924e70 ffff88042b924e70
[18273.732572] ffff88042b924e70 ffff88042d107d60 ffff88042d107de0 ffffc90003932180
[18273.741516] Call Trace:
[18273.744867] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18273.751016] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18273.758318] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18273.764848] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18273.771865] [<ffffffff810f53f2>] ? futex_wait_queue_me+0xa2/0x120
[18273.779362] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18273.785604] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18273.792800] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18273.799123] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18273.806420] automount S ffffc90003791700 0 1699 1 0x00000080
[18273.814907] ffff88042d16bcd0 0000000000000082 ffff88042b920000 ffff88042d16bfd8
[18273.823826] ffff88042d16bfd8 ffff88042d16bfd8 ffff88042b920000 ffff88042b920000
[18273.832740] ffff88042b920000 0000000000000000 ffff88042d16bde0 ffffc90003791700
[18273.841665] Call Trace:
[18273.845006] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18273.851167] [<ffffffff810f5414>] futex_wait_queue_me+0xc4/0x120
[18273.858501] [<ffffffff810f5f89>] futex_wait+0x179/0x280
[18273.865039] [<ffffffff810ca286>] ? sched_slice.isra.55+0x66/0xc0
[18273.872447] [<ffffffff810f5258>] ? get_futex_key+0x1c8/0x2c0
[18273.879486] [<ffffffff810f808e>] do_futex+0xfe/0x5b0
[18273.885734] [<ffffffff810c4e8c>] ? wake_up_new_task+0x10c/0x170
[18273.893049] [<ffffffff81084ec8>] ? do_fork+0xe8/0x2c0
[18273.899396] [<ffffffff810f85c0>] SyS_futex+0x80/0x180
[18273.905739] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18273.913042] automount S ffff880418ffbbb4 0 1746 1 0x00000080
[18273.921558] ffff880418ffba30 0000000000000082 ffff88042bd03ec0 ffff880418ffbfd8
[18273.930474] ffff880418ffbfd8 ffff880418ffbfd8 ffff88042bd03ec0 0000000000000000
[18273.939402] ffff880418ffbc90 00000000fffffffc 0000000000000000 ffff880418ffbbb4
[18273.948344] Call Trace:
[18273.951694] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18273.957845] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18273.966341] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18273.973279] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18273.980987] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18273.988406] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18273.995044] [<ffffffff812184b0>] ? __d_lookup+0x120/0x160
[18274.001776] [<ffffffff8121ef74>] ? mntput+0x24/0x40
[18274.007920] [<ffffffff81209851>] ? terminate_walk+0x51/0x60
[18274.014839] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18274.021958] [<ffffffff810d060c>] ? enqueue_entity+0x26c/0xb60
[18274.029075] [<ffffffff810d1108>] ? enqueue_task_fair+0x208/0x6c0
[18274.036493] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.044780] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.053042] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.061302] [<ffffffff81180b9b>] ? unlock_page+0x2b/0x30
[18274.067906] [<ffffffff811acd44>] ? do_read_fault.isra.42+0xe4/0x130
[18274.075591] [<ffffffff811b1431>] ? handle_mm_fault+0x6b1/0xfe0
[18274.082772] [<ffffffff810f8102>] ? do_futex+0x172/0x5b0
[18274.089275] [<ffffffff816928a1>] ? __do_page_fault+0x171/0x450
[18274.096462] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18274.102672] [<ffffffff8168edc8>] ? page_fault+0x28/0x30
[18274.109183] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18274.116469] automount S ffff88042ae1fbb4 0 1791 1 0x00000080
[18274.124943] ffff88042ae1fa30 0000000000000082 ffff88042bacde20 ffff88042ae1ffd8
[18274.133852] ffff88042ae1ffd8 ffff88042ae1ffd8 ffff88042bacde20 0000000000000000
[18274.142755] ffff88042ae1fc90 00000000fffffffc 0000000000000000 ffff88042ae1fbb4
[18274.151670] Call Trace:
[18274.154987] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18274.161116] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18274.169585] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18274.176503] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18274.184190] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18274.191599] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18274.198200] [<ffffffff812184b0>] ? __d_lookup+0x120/0x160
[18274.204909] [<ffffffff8121ef74>] ? mntput+0x24/0x40
[18274.211031] [<ffffffff81209851>] ? terminate_walk+0x51/0x60
[18274.217932] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18274.225025] [<ffffffff810d060c>] ? enqueue_entity+0x26c/0xb60
[18274.232112] [<ffffffff811a5064>] ? pcpu_alloc+0x354/0x640
[18274.238809] [<ffffffff810d1108>] ? enqueue_task_fair+0x208/0x6c0
[18274.246189] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.254460] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.262715] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.270945] [<ffffffff810f54d6>] ? wake_futex+0x66/0x80
[18274.277438] [<ffffffff810f5982>] ? futex_wake_op+0x332/0x570
[18274.284403] [<ffffffff810f8102>] ? do_futex+0x172/0x5b0
[18274.290872] [<ffffffff8128bf23>] ? autofs_dev_ioctl+0x13/0x20
[18274.297914] [<ffffffff81212555>] ? do_vfs_ioctl+0x2d5/0x4b0
[18274.304763] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18274.310930] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18274.318566] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18274.325801] automount S ffff8804140f7bb4 0 1825 1 0x00000080
[18274.334249] ffff8804140f7a30 0000000000000082 ffff8800360d6dd0 ffff8804140f7fd8
[18274.343117] ffff8804140f7fd8 ffff8804140f7fd8 ffff8800360d6dd0 0000000000000000
[18274.351986] ffff8804140f7c90 00000000fffffffc 0000000000000000 ffff8804140f7bb4
[18274.360844] Call Trace:
[18274.364124] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18274.370244] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18274.378690] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18274.385588] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18274.393273] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18274.400662] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18274.407286] [<ffffffff81033669>] ? sched_clock+0x9/0x10
[18274.413800] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18274.420789] [<ffffffff810295ec>] ? __switch_to+0x15c/0x4c0
[18274.427594] [<ffffffff81321c24>] ? timerqueue_del+0x24/0x70
[18274.434494] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.442756] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.451009] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.459263] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18274.466247] [<ffffffff810f5258>] ? get_futex_key+0x1c8/0x2c0
[18274.473213] [<ffffffff810f5570>] ? futex_wake+0x80/0x160
[18274.479785] [<ffffffff810f80b2>] ? do_futex+0x122/0x5b0
[18274.486256] [<ffffffff810c4e8c>] ? wake_up_new_task+0x10c/0x170
[18274.493502] [<ffffffff81084ec8>] ? do_fork+0xe8/0x2c0
[18274.499769] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18274.505933] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18274.513565] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18274.520797] automount S ffff8800b3f47bb4 0 1896 1 0x00000080
[18274.529247] ffff8800b3f47a30 0000000000000082 ffff88042a455e20 ffff8800b3f47fd8
[18274.538097] ffff8800b3f47fd8 ffff8800b3f47fd8 ffff88042a455e20 0000000000000000
[18274.546955] ffff8800b3f47c90 00000000fffffffc 0000000000000000 ffff8800b3f47bb4
[18274.555821] Call Trace:
[18274.559104] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18274.565214] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18274.573654] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18274.580551] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18274.588218] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18274.595598] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18274.602197] [<ffffffff812184b0>] ? __d_lookup+0x120/0x160
[18274.608895] [<ffffffff8121ef74>] ? mntput+0x24/0x40
[18274.615005] [<ffffffff81209851>] ? terminate_walk+0x51/0x60
[18274.621902] [<ffffffff810c9ba8>] ? __enqueue_entity+0x78/0x80
[18274.628993] [<ffffffff810d060c>] ? enqueue_entity+0x26c/0xb60
[18274.636081] [<ffffffff810d1108>] ? enqueue_task_fair+0x208/0x6c0
[18274.643466] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.651723] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.659970] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.668213] [<ffffffff810f54d6>] ? wake_futex+0x66/0x80
[18274.674688] [<ffffffff810f5982>] ? futex_wake_op+0x332/0x570
[18274.681645] [<ffffffff810f8102>] ? do_futex+0x172/0x5b0
[18274.688116] [<ffffffff8128bf23>] ? autofs_dev_ioctl+0x13/0x20
[18274.695169] [<ffffffff81212555>] ? do_vfs_ioctl+0x2d5/0x4b0
[18274.702017] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18274.708182] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18274.715797] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18274.723037] automount S ffff8801726f7ba4 0 8921 1 0x00000080
[18274.731472] ffff8801726f7a30 0000000000000082 ffff8801710d0fb0 ffff8801726f7fd8
[18274.740336] ffff8801726f7fd8 ffff8801726f7fd8 ffff8801710d0fb0 0000000000000000
[18274.749199] ffff8801726f7c90 00000000fffffffc 0000000000000000 ffff8801726f7ba4
[18274.758048] Call Trace:
[18274.761330] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18274.767429] [<ffffffff8168b4cd>] schedule_hrtimeout_range_clock+0x12d/0x150
[18274.775862] [<ffffffff810b16cc>] ? add_wait_queue+0x3c/0x50
[18274.782733] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18274.790405] [<ffffffff81213215>] poll_schedule_timeout+0x55/0xb0
[18274.797791] [<ffffffff8121479d>] do_sys_poll+0x4cd/0x580
[18274.804407] [<ffffffff811efdde>] ? memcg_check_events+0xbe/0x200
[18274.811793] [<ffffffff81213130>] ? poll_select_copy_remaining+0x150/0x150
[18274.820058] [<ffffffff8118ff0e>] ? lru_cache_add+0xe/0x10
[18274.826765] [<ffffffff811bb381>] ? page_add_new_anon_rmap+0x91/0x150
[18274.834561] [<ffffffff811ad706>] ? wp_page_copy.isra.56+0x2e6/0x470
[18274.842260] [<ffffffff811af215>] ? do_wp_page+0xe5/0x530
[18274.848885] [<ffffffff81318345>] ? cpumask_next_and+0x35/0x50
[18274.855990] [<ffffffff811b14ac>] ? handle_mm_fault+0x72c/0xfe0
[18274.863200] [<ffffffff8109abab>] ? recalc_sigpending+0x1b/0x50
[18274.870402] [<ffffffff8109b501>] ? __set_task_blocked+0x41/0xa0
[18274.877684] [<ffffffff81214954>] SyS_poll+0x74/0x110
[18274.883898] [<ffffffff8111f426>] ? __audit_syscall_exit+0x1e6/0x280
[18274.891575] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18274.898850] kworker/30:1H S ffff88082db96840 0 1758 2 0x00000080
[18274.907328] ffff8808273abe50 0000000000000046 ffff88082d6f0fb0 ffff8808273abfd8
[18274.916233] ffff8808273abfd8 ffff8808273abfd8 ffff88082d6f0fb0 ffff88082db96860
[18274.925110] ffff88042bd16530 ffff88082d6f0fb0 ffff88042bd16500 ffff88082db96840
[18274.933991] Call Trace:
[18274.937299] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18274.943428] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18274.950230] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18274.957330] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18274.963360] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18274.971242] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18274.977857] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18274.985741] master S ffff8800b2c07ea8 0 2314 1 0x00000080
[18274.994239] ffff8800b2c07db8 0000000000000086 ffff88042be69f60 ffff8800b2c07fd8
[18275.003171] ffff8800b2c07fd8 ffff8800b2c07fd8 ffff88042be69f60 ffff8800b2c07ea8
[18275.012107] 00000000039386ff 0000000000000000 ffff88042be69f60 ffff8800b2c07ea8
[18275.021034] Call Trace:
[18275.024389] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.030558] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18275.038963] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18275.046013] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18275.054630] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18275.062374] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18275.068659] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18275.075514] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18275.082364] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18275.089701] qmgr S ffff88082d6b7ea8 0 2355 2314 0x00000080
[18275.098237] ffff88082d6b7db8 0000000000000086 ffff88082a60ce70 ffff88082d6b7fd8
[18275.107180] ffff88082d6b7fd8 ffff88082d6b7fd8 ffff88082a60ce70 ffff88082d6b7ea8
[18275.116126] 0000000005f5e100 0000000000000000 ffff88082a60ce70 ffff88082d6b7ea8
[18275.125050] Call Trace:
[18275.128392] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.134561] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18275.142961] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18275.150005] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18275.158605] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18275.166328] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18275.172595] [<ffffffff81212555>] ? do_vfs_ioctl+0x2d5/0x4b0
[18275.179548] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18275.186390] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18275.193238] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18275.200578] gmond S ffff88082d5ffea8 0 2370 1 0x00000080
[18275.209127] ffff88082d5ffdb8 0000000000000086 ffff880826f8ce70 ffff88082d5fffd8
[18275.218073] ffff88082d5fffd8 ffff88082d5fffd8 ffff880826f8ce70 ffff88082d5ffea8
[18275.227009] 00000000000f423f 0000000000000000 ffff880826f8ce70 ffff88082d5ffea8
[18275.235946] Call Trace:
[18275.239319] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.245502] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18275.253912] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18275.260959] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18275.269570] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18275.277311] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18275.283589] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18275.290448] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18275.297301] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18275.304646] gmond S ffff8808242c3ea8 0 2464 1 0x00000080
[18275.313211] ffff8808242c3db8 0000000000000086 ffff88042bae5e20 ffff8808242c3fd8
[18275.322182] ffff8808242c3fd8 ffff8808242c3fd8 ffff88042bae5e20 ffff8808242c3ea8
[18275.331137] 000000000001869f 0000000000000000 ffff88042bae5e20 ffff8808242c3ea8
[18275.340083] Call Trace:
[18275.343462] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.349642] [<ffffffff8168b452>] schedule_hrtimeout_range_clock+0xb2/0x150
[18275.358055] [<ffffffff810b4640>] ? hrtimer_get_res+0x50/0x50
[18275.365119] [<ffffffff8168b446>] ? schedule_hrtimeout_range_clock+0xa6/0x150
[18275.373744] [<ffffffff8168b503>] schedule_hrtimeout_range+0x13/0x20
[18275.381480] [<ffffffff8124873e>] ep_poll+0x23e/0x360
[18275.387754] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18275.394628] [<ffffffff812499ad>] SyS_epoll_wait+0xed/0x120
[18275.401491] [<ffffffff816973c9>] system_call_fastpath+0x16/0x1b
[18275.408821] kworker/23:1H S ffff88042e3d6840 0 2924 2 0x00000080
[18275.417381] ffff88042b323e50 0000000000000046 ffff88042bb10fb0 ffff88042b323fd8
[18275.426354] ffff88042b323fd8 ffff88042b323fd8 ffff88042bb10fb0 ffff88042e3d6860
[18275.435326] ffff88042bc2d1b0 ffff88042bb10fb0 ffff88042bc2d180 ffff88042e3d6840
[18275.444288] Call Trace:
[18275.447666] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.453836] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18275.460706] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18275.467855] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18275.473925] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.481834] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18275.488498] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.496417] kworker/23:2 S ffff88042e3d6480 0 2972 2 0x00000080
[18275.504953] ffff880412e6fe50 0000000000000046 ffff88042bacce70 ffff880412e6ffd8
[18275.513911] ffff880412e6ffd8 ffff880412e6ffd8 ffff88042bacce70 ffff88042e3d64a0
[18275.522854] ffff88042bc2cfb0 ffff88042bacce70 ffff88042bc2cf80 ffff88042e3d6480
[18275.531804] Call Trace:
[18275.535177] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.541376] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18275.548243] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18275.555398] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18275.561476] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.569405] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18275.576053] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.583965] kworker/u484:1 S ffff88082a749000 0 3054 2 0x00000080
[18275.592507] ffff880412b73e50 0000000000000046 ffff880416300000 ffff880412b73fd8
[18275.601477] ffff880412b73fd8 ffff880412b73fd8 ffff880416300000 ffff88082a749020
[18275.610437] ffff88042c261230 ffff880416300000 ffff88042c261200 ffff88082a749000
[18275.619395] Call Trace:
[18275.622751] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.628919] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18275.635766] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18275.642916] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18275.648988] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.656905] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18275.663560] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.671485] kworker/3:3 S ffff88042e0d6480 0 3056 2 0x00000080
[18275.680007] ffff880412f1fe50 0000000000000046 ffff880416301f60 ffff880412f1ffd8
[18275.688945] ffff880412f1ffd8 ffff880412f1ffd8 ffff880416301f60 ffff88042e0d64a0
[18275.697898] ffff88042fce3830 ffff880416301f60 ffff88042fce3800 ffff88042e0d6480
[18275.706847] Call Trace:
[18275.710201] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.716375] [<ffffffff810a9349>] worker_thread+0x1d9/0x410
[18275.723233] [<ffffffff810a9170>] ? rescuer_thread+0x460/0x460
[18275.730381] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18275.736458] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.744382] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18275.751036] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.758948] spl_kmem_cache S ffff88081e697700 0 3082 2 0x00000080
[18275.767487] ffff8808257fbe08 0000000000000046 ffff880416305e20 ffff8808257fbfd8
[18275.776444] ffff8808257fbfd8 ffff8808257fbfd8 ffff880416305e20 ffff88042bc131e0
[18275.785416] ffff88081e697790 ffffffffa08254b0 ffff88081e6977b0 ffff88081e697700
[18275.794366] Call Trace:
[18275.797740] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18275.805686] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.811875] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18275.819238] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18275.826106] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18275.834037] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18275.840134] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.848066] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18275.854733] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.862661] spl_kmem_cache S ffff88081e697700 0 3083 2 0x00000080
[18275.871222] ffff880825777e08 0000000000000046 ffff880416306dd0 ffff880825777fd8
[18275.880194] ffff880825777fd8 ffff880825777fd8 ffff880416306dd0 ffff88042bc13240
[18275.889168] ffff88081e697790 ffffffffa08254b0 ffff88081e6977b0 ffff88081e697700
[18275.898140] Call Trace:
[18275.901521] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18275.909460] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18275.915677] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18275.923048] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18275.929934] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18275.937880] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18275.943974] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.951917] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18275.958605] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18275.966547] spl_kmem_cache S ffff88081e697700 0 3084 2 0x00000080
[18275.975116] ffff88082570be08 0000000000000046 ffff880416304e70 ffff88082570bfd8
[18275.984098] ffff88082570bfd8 ffff88082570bfd8 ffff880416304e70 ffff88042bc132a0
[18275.993071] ffff88081e697790 ffffffffa08254b0 ffff88081e6977b0 ffff88081e697700
[18276.002057] Call Trace:
[18276.005471] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.013440] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.019667] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.027042] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.033930] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.041887] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.048001] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.055957] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.062646] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.070608] spl_kmem_cache S ffff88081e697700 0 3085 2 0x00000080
[18276.079199] ffff88082a39fe08 0000000000000046 ffff880416303ec0 ffff88082a39ffd8
[18276.088205] ffff88082a39ffd8 ffff88082a39ffd8 ffff880416303ec0 ffff88042bc13300
[18276.097221] ffff88081e697790 ffffffffa08254b0 ffff88081e6977b0 ffff88081e697700
[18276.106210] Call Trace:
[18276.109631] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.117624] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.123845] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.131238] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.138149] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.146116] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.152236] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.160211] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.166916] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.174882] spl_system_task S ffff88081e695e00 0 3086 2 0x00000080
[18276.183471] ffff8808273dbe08 0000000000000046 ffff88082c89ce70 ffff8808273dbfd8
[18276.192469] ffff8808273dbfd8 ffff8808273dbfd8 ffff88082c89ce70 ffff88042bc13360
[18276.201472] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18276.210477] Call Trace:
[18276.213896] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.221866] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.228088] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.235486] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.242389] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.250354] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.256465] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.264426] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.271118] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.279073] spl_system_task S ffff88081e695e00 0 3087 2 0x00000080
[18276.287670] ffff8808273ebe08 0000000000000046 ffff88082c89bec0 ffff8808273ebfd8
[18276.296687] ffff8808273ebfd8 ffff8808273ebfd8 ffff88082c89bec0 ffff88042bc133c0
[18276.305698] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18276.314700] Call Trace:
[18276.318111] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.326085] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.332307] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.339709] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.346625] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.354594] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.360710] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.368668] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.375363] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.383322] spl_system_task S ffff88081e695e00 0 3088 2 0x00000080
[18276.391905] ffff880823383e08 0000000000000046 ffff88082a405e20 ffff880823383fd8
[18276.400909] ffff880823383fd8 ffff880823383fd8 ffff88082a405e20 ffff88042bc13420
[18276.409908] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18276.418899] Call Trace:
[18276.422306] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.430284] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.436502] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.443889] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.450806] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.458784] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.464897] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.472854] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.479564] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.487528] spl_system_task S ffff88081e695e00 0 3089 2 0x00000080
[18276.496097] ffff88083fd37e08 0000000000000046 ffff88082a402f10 ffff88083fd37fd8
[18276.505092] ffff88083fd37fd8 ffff88083fd37fd8 ffff88082a402f10 ffff88042bc13480
[18276.514082] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18276.523080] Call Trace:
[18276.526493] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.534468] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.540697] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.548086] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.554994] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.562963] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.569075] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.577033] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.583744] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.591709] spl_system_task S ffff88081e695e00 0 3090 2 0x00000080
[18276.600286] ffff88082d4efe08 0000000000000046 ffff88082a404e70 ffff88082d4effd8
[18276.609291] ffff88082d4effd8 ffff88082d4effd8 ffff88082a404e70 ffff88042bc134e0
[18276.618285] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18276.627284] Call Trace:
[18276.630697] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.638671] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.644890] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.652286] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.659204] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.667168] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.673273] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.681237] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.687934] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.695892] spl_system_task S ffff88081e695e00 0 3091 2 0x00000080
[18276.704471] ffff88082739be08 0000000000000046 ffff88082a406dd0 ffff88082739bfd8
[18276.713476] ffff88082739bfd8 ffff88082739bfd8 ffff88082a406dd0 ffff88042bc13540
[18276.722475] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18276.731468] Call Trace:
[18276.734875] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.742848] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.749066] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.756460] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.763363] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.771336] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.777453] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.785421] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.792112] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.800064] spl_system_task S ffff88081e695e00 0 3092 2 0x00000080
[18276.808655] ffff880826267e08 0000000000000046 ffff88082a403ec0 ffff880826267fd8
[18276.817660] ffff880826267fd8 ffff880826267fd8 ffff88082a403ec0 ffff88042bc135a0
[18276.826661] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18276.835669] Call Trace:
[18276.839080] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.847054] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.853274] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.860675] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.867577] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.875542] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.881664] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.889629] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18276.896330] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.904294] spl_system_task S ffff88081e695e00 0 3093 2 0x00000080
[18276.912866] ffff8808253b3e08 0000000000000046 ffff880824a7edd0 ffff8808253b3fd8
[18276.921870] ffff8808253b3fd8 ffff8808253b3fd8 ffff880824a7edd0 ffff88042bc13600
[18276.930844] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18276.939843] Call Trace:
[18276.943265] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.951241] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18276.957463] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18276.964852] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18276.971761] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18276.979726] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18276.985831] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18276.993800] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.000496] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.008457] spl_system_task S ffff88081e695e00 0 3094 2 0x00000080
[18277.017019] ffff88081e70be08 0000000000000046 ffff880824a7de20 ffff88081e70bfd8
[18277.026019] ffff88081e70bfd8 ffff88081e70bfd8 ffff880824a7de20 ffff88042bc13660
[18277.035019] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.044006] Call Trace:
[18277.047414] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.055392] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.061612] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.068998] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.075902] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.083857] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.089969] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.097926] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.104628] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.112594] spl_system_task S ffff88081e695e00 0 3095 2 0x00000080
[18277.121174] ffff88082dbffe08 0000000000000046 ffff880824a7ce70 ffff88082dbfffd8
[18277.130177] ffff88082dbfffd8 ffff88082dbfffd8 ffff880824a7ce70 ffff88042bc136c0
[18277.139178] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.148179] Call Trace:
[18277.151583] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.159563] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.165793] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.173177] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.180070] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.188025] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.194132] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.202088] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.208792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.216756] spl_system_task S ffff88081e695e00 0 3096 2 0x00000080
[18277.225349] ffff88081e6d7e08 0000000000000046 ffff880824a7bec0 ffff88081e6d7fd8
[18277.234339] ffff88081e6d7fd8 ffff88081e6d7fd8 ffff880824a7bec0 ffff88042bc13720
[18277.243331] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.252334] Call Trace:
[18277.255759] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.263739] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.269955] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.277351] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.284248] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.292210] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.298321] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.306275] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.312970] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.320921] spl_system_task S ffff88081e695e00 0 3097 2 0x00000080
[18277.329507] ffff88082ab47e08 0000000000000046 ffff880824a7af10 ffff88082ab47fd8
[18277.338519] ffff88082ab47fd8 ffff88082ab47fd8 ffff880824a7af10 ffff88042bc13780
[18277.347528] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.356526] Call Trace:
[18277.359929] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.367891] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.374107] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.381494] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.388404] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.396374] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.402484] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.410442] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.417136] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.425092] spl_system_task S ffff88081e695e00 0 3098 2 0x00000080
[18277.433678] ffff880826b6fe08 0000000000000046 ffff880824a79f60 ffff880826b6ffd8
[18277.442678] ffff880826b6ffd8 ffff880826b6ffd8 ffff880824a79f60 ffff88042bc137e0
[18277.451675] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.460675] Call Trace:
[18277.464084] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.472052] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.478265] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.485659] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.492568] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.500538] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.506658] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.514615] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.521295] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.529265] spl_system_task S ffff88081e695e00 0 3099 2 0x00000080
[18277.537845] ffff880824ba3e08 0000000000000046 ffff880824a78fb0 ffff880824ba3fd8
[18277.546845] ffff880824ba3fd8 ffff880824ba3fd8 ffff880824a78fb0 ffff88042bc13840
[18277.555833] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.564842] Call Trace:
[18277.568250] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.576225] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.582441] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.589839] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.596746] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.604708] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.610834] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.618794] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.625496] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.633457] spl_system_task S ffff88081e695e00 0 3100 2 0x00000080
[18277.642021] ffff88082d9ffe08 0000000000000046 ffff880824a78000 ffff88082d9fffd8
[18277.651020] ffff88082d9fffd8 ffff88082d9fffd8 ffff880824a78000 ffff88042bc138a0
[18277.660020] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.669013] Call Trace:
[18277.672421] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.680391] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.686618] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.694000] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.700898] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.708863] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.714973] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.722931] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.729629] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.737590] spl_system_task S ffff88081e695e00 0 3101 2 0x00000080
[18277.746156] ffff8808276afe08 0000000000000046 ffff88082a638fb0 ffff8808276affd8
[18277.755129] ffff8808276affd8 ffff8808276affd8 ffff88082a638fb0 ffff88042bc13900
[18277.764127] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.773120] Call Trace:
[18277.776531] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.784508] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.790725] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.798109] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.805007] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.812953] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.819061] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.827016] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.833698] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.841654] spl_system_task S ffff88081e695e00 0 3102 2 0x00000080
[18277.850243] ffff8808266d7e08 0000000000000046 ffff88082a63de20 ffff8808266d7fd8
[18277.859244] ffff8808266d7fd8 ffff8808266d7fd8 ffff88082a63de20 ffff88042bc13960
[18277.868230] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.877228] Call Trace:
[18277.880647] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.888631] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.894853] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18277.902241] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18277.909142] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.917104] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18277.923223] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.931178] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18277.937871] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18277.945836] spl_system_task S ffff88081e695e00 0 3103 2 0x00000080
[18277.954407] ffff88082daffe08 0000000000000046 ffff88082a63af10 ffff88082dafffd8
[18277.963412] ffff88082dafffd8 ffff88082dafffd8 ffff88082a63af10 ffff88042bc139c0
[18277.972406] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18277.981409] Call Trace:
[18277.984837] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18277.992821] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18277.999039] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.006425] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.013332] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.021299] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.027425] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.035391] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.042086] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.050036] spl_system_task S ffff88081e695e00 0 3104 2 0x00000080
[18278.058635] ffff88082c7e3e08 0000000000000046 ffff88082d6f6dd0 ffff88082c7e3fd8
[18278.067647] ffff88082c7e3fd8 ffff88082c7e3fd8 ffff88082d6f6dd0 ffff88042bc13a20
[18278.076660] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.085660] Call Trace:
[18278.089074] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.097042] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.103260] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.110657] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.117562] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.125527] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.131632] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.139590] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.146296] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.154256] spl_system_task S ffff88081e695e00 0 3105 2 0x00000080
[18278.162838] ffff880829af7e08 0000000000000046 ffff88082d6f1f60 ffff880829af7fd8
[18278.171844] ffff880829af7fd8 ffff880829af7fd8 ffff88082d6f1f60 ffff88042bc13a80
[18278.180843] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.189849] Call Trace:
[18278.193267] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.201246] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.207465] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.214848] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.221758] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.229736] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.235858] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.243832] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.250537] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.258505] spl_system_task S ffff88081e695e00 0 3106 2 0x00000080
[18278.267073] ffff88082aa93e08 0000000000000046 ffff88082d6f3ec0 ffff88082aa93fd8
[18278.276076] ffff88082aa93fd8 ffff88082aa93fd8 ffff88082d6f3ec0 ffff88042bc13ae0
[18278.285079] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.294074] Call Trace:
[18278.297486] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.305459] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.311679] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.319064] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.325958] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.333915] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.340031] [<ffffffff810bfde3>] ? finish_task_switch+0x53/0x180
[18278.347502] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.355466] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.362151] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.370109] spl_system_task S ffff88081e695e00 0 3107 2 0x00000080
[18278.378677] ffff880826acbe08 0000000000000046 ffff88082d6f5e20 ffff880826acbfd8
[18278.387680] ffff880826acbfd8 ffff880826acbfd8 ffff88082d6f5e20 ffff88042bc13b40
[18278.396684] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.405683] Call Trace:
[18278.409074] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.417037] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.423254] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.430648] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.437550] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.445516] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.451637] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.459593] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.466293] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.474255] spl_system_task S ffff88081e695e00 0 3108 2 0x00000080
[18278.482831] ffff880827b9be08 0000000000000046 ffff88082d6f2f10 ffff880827b9bfd8
[18278.491824] ffff880827b9bfd8 ffff880827b9bfd8 ffff88082d6f2f10 ffff88042bc13ba0
[18278.500835] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.509831] Call Trace:
[18278.513232] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.521191] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.527412] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.534798] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.541695] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.549662] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.555785] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.563739] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.570434] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.578391] spl_system_task S ffff88081e695e00 0 3109 2 0x00000080
[18278.586957] ffff880826b63e08 0000000000000046 ffff88082d6f0000 ffff880826b63fd8
[18278.595952] ffff880826b63fd8 ffff880826b63fd8 ffff88082d6f0000 ffff88042bc13c00
[18278.604940] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.613945] Call Trace:
[18278.617349] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.625316] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.631537] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.638921] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.645815] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.653772] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.659885] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.667839] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.674540] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.682504] spl_system_task S ffff88081e695e00 0 3110 2 0x00000080
[18278.691073] ffff880827bafe08 0000000000000046 ffff88082bcb3ec0 ffff880827baffd8
[18278.700065] ffff880827baffd8 ffff880827baffd8 ffff88082bcb3ec0 ffff88042bc13c60
[18278.709060] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.718051] Call Trace:
[18278.721461] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.729440] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.735665] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.743050] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.749959] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.757916] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.764030] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.771980] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.778674] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.786636] spl_system_task S ffff88081e695e00 0 3111 2 0x00000080
[18278.795207] ffff880827657e08 0000000000000046 ffff88082bcb0fb0 ffff880827657fd8
[18278.804185] ffff880827657fd8 ffff880827657fd8 ffff88082bcb0fb0 ffff88042bc13cc0
[18278.813176] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.822169] Call Trace:
[18278.825584] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.833557] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.839789] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.847170] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.854071] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.862040] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.868151] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.876109] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.882814] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.890782] spl_system_task S ffff88081e695e00 0 3112 2 0x00000080
[18278.899369] ffff88082ab4be08 0000000000000046 ffff88082e5b8000 ffff88082ab4bfd8
[18278.908369] ffff88082ab4bfd8 ffff88082ab4bfd8 ffff88082e5b8000 ffff88042bc13d20
[18278.917365] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18278.926363] Call Trace:
[18278.929796] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.937779] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18278.943998] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18278.951386] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18278.958291] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18278.966267] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18278.972378] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.980334] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18278.987036] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18278.994994] spl_system_task S ffff88081e695e00 0 3113 2 0x00000080
[18279.003577] ffff880824bbfe08 0000000000000046 ffff88082e5b9f60 ffff880824bbffd8
[18279.012580] ffff880824bbffd8 ffff880824bbffd8 ffff88082e5b9f60 ffff88042bc13d80
[18279.021584] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.030580] Call Trace:
[18279.033988] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.041954] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.048172] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.055566] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.062467] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.070429] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.076549] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.084506] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.091193] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.099145] spl_system_task S ffff88081e695e00 0 3114 2 0x00000080
[18279.107738] ffff88081e753e08 0000000000000046 ffff88082e5baf10 ffff88081e753fd8
[18279.116750] ffff88081e753fd8 ffff88081e753fd8 ffff88082e5baf10 ffff88042bc13de0
[18279.125754] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.134758] Call Trace:
[18279.138167] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.146138] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.152366] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.159762] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.166662] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.174623] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.180745] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.188708] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.195399] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.203353] spl_system_task S ffff88081e695e00 0 3115 2 0x00000080
[18279.211932] ffff880824317e08 0000000000000046 ffff88082e5bbec0 ffff880824317fd8
[18279.220933] ffff880824317fd8 ffff880824317fd8 ffff88082e5bbec0 ffff88042bc13e40
[18279.229929] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.238919] Call Trace:
[18279.242337] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.250307] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.256540] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.263925] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.270827] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.278790] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.284908] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.292872] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.299566] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.307518] spl_system_task S ffff88081e695e00 0 3116 2 0x00000080
[18279.316088] ffff88082475be08 0000000000000046 ffff88082e5bce70 ffff88082475bfd8
[18279.325082] ffff88082475bfd8 ffff88082475bfd8 ffff88082e5bce70 ffff88042bc13ea0
[18279.334076] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.343072] Call Trace:
[18279.346490] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.354459] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.360688] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.368071] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.374970] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.382928] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.389036] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.396990] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.403692] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.411659] spl_system_task S ffff88081e695e00 0 3117 2 0x00000080
[18279.420229] ffff880827b8fe08 0000000000000046 ffff88082e5bedd0 ffff880827b8ffd8
[18279.429212] ffff880827b8ffd8 ffff880827b8ffd8 ffff88082e5bedd0 ffff88042bc13f00
[18279.438196] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.447186] Call Trace:
[18279.450604] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.458585] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.464809] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.472194] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.479094] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.487053] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.493162] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.501117] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.507824] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.515781] spl_system_task S ffff88081e695e00 0 3118 2 0x00000080
[18279.524369] ffff88082d2e3e08 0000000000000046 ffff88082e5bde20 ffff88082d2e3fd8
[18279.533368] ffff88082d2e3fd8 ffff88082d2e3fd8 ffff88082e5bde20 ffff88042bc13f60
[18279.542377] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.551371] Call Trace:
[18279.554788] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.562773] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.568989] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.576383] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.583283] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.591238] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.597355] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.605321] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.612008] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.619965] spl_system_task S ffff88081e695e00 0 3119 2 0x00000080
[18279.628561] ffff88082aec3e08 0000000000000046 ffff88082d5c0000 ffff88082aec3fd8
[18279.637568] ffff88082aec3fd8 ffff88082aec3fd8 ffff88082d5c0000 ffff88042bc130c0
[18279.646563] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.655569] Call Trace:
[18279.658974] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.666945] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.673158] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.680544] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.687446] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.695405] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.701532] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.709496] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.716182] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.724136] spl_system_task S ffff88081e695e00 0 3120 2 0x00000080
[18279.732735] ffff88082676fe08 0000000000000046 ffff88082d5c0fb0 ffff88082676ffd8
[18279.741725] ffff88082676ffd8 ffff88082676ffd8 ffff88082d5c0fb0 ffff88042bc13060
[18279.750709] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.759712] Call Trace:
[18279.763124] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.771096] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.777312] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.784704] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.791604] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.799574] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.805697] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.813656] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.820350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.828308] spl_system_task S ffff88081e695e00 0 3121 2 0x00000080
[18279.836895] ffff880825fd7e08 0000000000000046 ffff88082d5c1f60 ffff880825fd7fd8
[18279.845894] ffff880825fd7fd8 ffff880825fd7fd8 ffff88082d5c1f60 ffff88042bcad480
[18279.854899] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.863900] Call Trace:
[18279.867308] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.875283] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.881504] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.888905] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.895821] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.903788] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18279.909906] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.917871] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18279.924574] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18279.932543] spl_system_task S ffff88081e695e00 0 3122 2 0x00000080
[18279.941114] ffff88082664be08 0000000000000046 ffff88082d5c2f10 ffff88082664bfd8
[18279.950110] ffff88082664bfd8 ffff88082664bfd8 ffff88082d5c2f10 ffff88042bcade40
[18279.959107] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18279.968103] Call Trace:
[18279.971519] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18279.979488] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18279.985708] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18279.993088] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18279.999982] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.007943] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.014057] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.022015] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.028708] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.036662] spl_system_task S ffff88081e695e00 0 3123 2 0x00000080
[18280.045231] ffff880827bb3e08 0000000000000046 ffff88082d5c3ec0 ffff880827bb3fd8
[18280.054221] ffff880827bb3fd8 ffff880827bb3fd8 ffff88082d5c3ec0 ffff88042bcad840
[18280.063210] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.072205] Call Trace:
[18280.075619] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.083591] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.089821] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.097208] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.104109] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.112086] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.118198] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.126149] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.132853] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.140810] spl_system_task S ffff88081e695e00 0 3124 2 0x00000080
[18280.149404] ffff880826abbe08 0000000000000046 ffff88082d5c4e70 ffff880826abbfd8
[18280.158407] ffff880826abbfd8 ffff880826abbfd8 ffff88082d5c4e70 ffff88042bcad120
[18280.167411] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.176409] Call Trace:
[18280.179814] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.187790] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.194005] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.201395] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.208312] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.216273] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.222388] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.230353] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.237055] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.245019] spl_system_task S ffff88081e695e00 0 3125 2 0x00000080
[18280.253616] ffff88082debfe08 0000000000000046 ffff88082d5c5e20 ffff88082debffd8
[18280.262622] ffff88082debffd8 ffff88082debffd8 ffff88082d5c5e20 ffff88042bcada80
[18280.271630] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.280646] Call Trace:
[18280.284055] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.292036] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.298254] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.305645] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.312543] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.320516] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.326638] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.334601] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.341302] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.349261] spl_system_task S ffff88081e695e00 0 3126 2 0x00000080
[18280.357842] ffff88082623be08 0000000000000046 ffff88082d5c6dd0 ffff88082623bfd8
[18280.366852] ffff88082623bfd8 ffff88082623bfd8 ffff88082d5c6dd0 ffff88042bcad2a0
[18280.375854] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.384855] Call Trace:
[18280.388268] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.396241] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.402450] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.409836] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.416748] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.424711] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.430835] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.438803] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.445494] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.453456] spl_system_task S ffff88081e695e00 0 3127 2 0x00000080
[18280.462031] ffff88082e0b3e08 0000000000000046 ffff88082aba0000 ffff88082e0b3fd8
[18280.471030] ffff88082e0b3fd8 ffff88082e0b3fd8 ffff88082aba0000 ffff88042bcad180
[18280.480029] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.489027] Call Trace:
[18280.492436] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.500416] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.506642] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.514028] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.520949] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.528922] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.535034] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.542992] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.549694] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.557659] spl_system_task S ffff88081e695e00 0 3128 2 0x00000080
[18280.566227] ffff88082e3b7e08 0000000000000046 ffff88082aba0fb0 ffff88082e3b7fd8
[18280.575229] ffff88082e3b7fd8 ffff88082e3b7fd8 ffff88082aba0fb0 ffff88042bcada20
[18280.584221] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.593225] Call Trace:
[18280.596643] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.604624] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.610854] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.618239] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.625140] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.633098] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.639204] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.647162] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.653868] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.661836] spl_system_task S ffff88081e695e00 0 3129 2 0x00000080
[18280.670424] ffff88082cc03e08 0000000000000046 ffff88082aba1f60 ffff88082cc03fd8
[18280.679420] ffff88082cc03fd8 ffff88082cc03fd8 ffff88082aba1f60 ffff88042bcadba0
[18280.688411] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.697421] Call Trace:
[18280.700825] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.708799] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.715031] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.722428] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.729332] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.737291] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.743411] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.751378] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.758064] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.766017] spl_system_task S ffff88081e695e00 0 3130 2 0x00000080
[18280.774598] ffff88082afb7e08 0000000000000046 ffff88082aba2f10 ffff88082afb7fd8
[18280.783610] ffff88082afb7fd8 ffff88082afb7fd8 ffff88082aba2f10 ffff88042bcad8a0
[18280.792625] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.801633] Call Trace:
[18280.805040] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.813012] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.819225] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.826622] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.833523] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.841490] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.847606] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.855574] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.862268] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.870227] spl_system_task S ffff88081e695e00 0 3131 2 0x00000080
[18280.878808] ffff88082431fe08 0000000000000046 ffff88082aba3ec0 ffff88082431ffd8
[18280.887819] ffff88082431ffd8 ffff88082431ffd8 ffff88082aba3ec0 ffff88042bcadea0
[18280.896820] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18280.905825] Call Trace:
[18280.909234] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.917209] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18280.923435] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18280.930832] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18280.937735] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18280.945699] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18280.951816] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.959782] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18280.966479] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18280.974439] spl_system_task S ffff88081e695e00 0 3132 2 0x00000080
[18280.983003] ffff880824323e08 0000000000000046 ffff88082aba4e70 ffff880824323fd8
[18280.992000] ffff880824323fd8 ffff880824323fd8 ffff88082aba4e70 ffff88042bcad420
[18281.000993] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.009990] Call Trace:
[18281.013409] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.021394] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.027616] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.035009] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.041920] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.049891] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.056005] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.063978] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.070674] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.078644] spl_system_task S ffff88081e695e00 0 3133 2 0x00000080
[18281.087226] ffff880829c6be08 0000000000000046 ffff88082aba5e20 ffff880829c6bfd8
[18281.096208] ffff880829c6bfd8 ffff880829c6bfd8 ffff88082aba5e20 ffff88042bcadc00
[18281.105198] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.114196] Call Trace:
[18281.117620] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.125604] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.131833] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.139213] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.146107] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.154064] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.160176] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.168134] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.174829] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.182785] spl_system_task S ffff88081e695e00 0 3134 2 0x00000080
[18281.191361] ffff88082af6fe08 0000000000000046 ffff88082aba6dd0 ffff88082af6ffd8
[18281.200362] ffff88082af6ffd8 ffff88082af6ffd8 ffff88082aba6dd0 ffff88042bcad600
[18281.209356] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.218347] Call Trace:
[18281.221748] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.229726] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.235959] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.243353] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.250254] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.258217] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.264340] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.272292] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.278987] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.286946] spl_system_task S ffff88081e695e00 0 3135 2 0x00000080
[18281.295522] ffff880826fffe08 0000000000000046 ffff880825780000 ffff880826ffffd8
[18281.304536] ffff880826ffffd8 ffff880826ffffd8 ffff880825780000 ffff88042bcad6c0
[18281.313537] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.322544] Call Trace:
[18281.325964] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.333937] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.340147] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.347536] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.354438] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.362400] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.368512] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.376470] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.383166] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.391122] spl_system_task S ffff88081e695e00 0 3136 2 0x00000080
[18281.399702] ffff88082db37e08 0000000000000046 ffff880825780fb0 ffff88082db37fd8
[18281.408715] ffff88082db37fd8 ffff88082db37fd8 ffff880825780fb0 ffff88042bcadcc0
[18281.417723] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.426723] Call Trace:
[18281.430131] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.438104] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.444322] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.451715] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.458615] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.466584] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.472705] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.480670] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.487365] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.495332] spl_system_task S ffff88081e695e00 0 3137 2 0x00000080
[18281.503913] ffff880829f0be08 0000000000000046 ffff880825781f60 ffff880829f0bfd8
[18281.512927] ffff880829f0bfd8 ffff880829f0bfd8 ffff880825781f60 ffff88042bcadb40
[18281.521924] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.530923] Call Trace:
[18281.534330] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.542301] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.548520] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.555909] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.562815] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.570776] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.576899] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.584865] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.591566] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.599531] spl_system_task S ffff88081e695e00 0 3138 2 0x00000080
[18281.608099] ffff88082c3cbe08 0000000000000046 ffff880825782f10 ffff88082c3cbfd8
[18281.617093] ffff88082c3cbfd8 ffff88082c3cbfd8 ffff880825782f10 ffff88042bcadde0
[18281.626094] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.635083] Call Trace:
[18281.638501] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.646473] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.652691] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.660077] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.666987] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.674959] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.681068] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.689024] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.695727] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.703693] spl_system_task S ffff88081e695e00 0 3139 2 0x00000080
[18281.712265] ffff8808266ebe08 0000000000000046 ffff880825783ec0 ffff8808266ebfd8
[18281.721261] ffff8808266ebfd8 ffff8808266ebfd8 ffff880825783ec0 ffff88042bcad7e0
[18281.730265] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.739255] Call Trace:
[18281.742659] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.750636] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.756864] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.764250] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.771151] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.779108] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.785211] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.793171] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.799873] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.807839] spl_system_task S ffff88081e695e00 0 3140 2 0x00000080
[18281.816430] ffff8808266c3e08 0000000000000046 ffff880825784e70 ffff8808266c3fd8
[18281.825431] ffff8808266c3fd8 ffff8808266c3fd8 ffff880825784e70 ffff88042bcad4e0
[18281.834436] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.843433] Call Trace:
[18281.846844] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.854835] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.861050] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.868435] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.875338] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.883298] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.889412] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.897384] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18281.904080] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18281.912030] spl_system_task S ffff88081e695e00 0 3141 2 0x00000080
[18281.920627] ffff88082e57fe08 0000000000000046 ffff880825785e20 ffff88082e57ffd8
[18281.929626] ffff88082e57ffd8 ffff88082e57ffd8 ffff880825785e20 ffff88042bcadae0
[18281.938627] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18281.947611] Call Trace:
[18281.951022] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.959006] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18281.965222] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18281.972617] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18281.979517] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18281.987486] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18281.993607] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.001564] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.008256] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.016211] spl_system_task S ffff88081e695e00 0 3142 2 0x00000080
[18282.024802] ffff88082bf03e08 0000000000000046 ffff880825786dd0 ffff88082bf03fd8
[18282.033816] ffff88082bf03fd8 ffff88082bf03fd8 ffff880825786dd0 ffff88042bcad300
[18282.042806] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18282.051803] Call Trace:
[18282.055215] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.063186] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.069418] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.076814] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.083714] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.091674] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.097794] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.105754] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.112442] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.120399] spl_system_task S ffff88081e695e00 0 3143 2 0x00000080
[18282.128991] ffff880827f5fe08 0000000000000046 ffff8808257c0000 ffff880827f5ffd8
[18282.137982] ffff880827f5ffd8 ffff880827f5ffd8 ffff8808257c0000 ffff88042bcad1e0
[18282.146982] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18282.155975] Call Trace:
[18282.159375] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.167338] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.173552] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.180952] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.187843] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.195810] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.201923] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.209872] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.216565] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.224534] spl_system_task S ffff88081e695e00 0 3144 2 0x00000080
[18282.233112] ffff880823e17e08 0000000000000046 ffff8808257c0fb0 ffff880823e17fd8
[18282.242095] ffff880823e17fd8 ffff880823e17fd8 ffff8808257c0fb0 ffff88042bcad9c0
[18282.251086] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18282.260086] Call Trace:
[18282.263494] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.271463] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.277685] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.285076] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.291976] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.299951] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.306059] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.314018] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.320719] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.328671] spl_system_task S ffff88081e695e00 0 3145 2 0x00000080
[18282.337235] ffff88081c33fe08 0000000000000046 ffff8808257c1f60 ffff88081c33ffd8
[18282.346225] ffff88081c33ffd8 ffff88081c33ffd8 ffff8808257c1f60 ffff88042bcad780
[18282.355226] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18282.364217] Call Trace:
[18282.367621] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.375592] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.381811] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.389193] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.396089] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.404049] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.410159] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.418114] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.424816] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.432773] spl_system_task S ffff88081e695e00 0 3146 2 0x00000080
[18282.441343] ffff8808296dfe08 0000000000000046 ffff8808257c2f10 ffff8808296dffd8
[18282.450338] ffff8808296dffd8 ffff8808296dffd8 ffff8808257c2f10 ffff88042bcad900
[18282.459321] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18282.468304] Call Trace:
[18282.471717] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.479701] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.485917] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.493294] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.500193] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.508153] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.514262] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.522219] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.528927] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.536886] spl_system_task S ffff88081e695e00 0 3147 2 0x00000080
[18282.545470] ffff88082d83be08 0000000000000046 ffff8808257c3ec0 ffff88082d83bfd8
[18282.554471] ffff88082d83bfd8 ffff88082d83bfd8 ffff8808257c3ec0 ffff88042bcadf00
[18282.563461] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18282.572452] Call Trace:
[18282.575865] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.583843] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.590060] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.597444] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.604342] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.612296] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.618415] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.626381] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.633081] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.641037] spl_system_task S ffff88081e695e00 0 3148 2 0x00000080
[18282.649627] ffff88082432be08 0000000000000046 ffff8808257c4e70 ffff88082432bfd8
[18282.658636] ffff88082432bfd8 ffff88082432bfd8 ffff8808257c4e70 ffff88042bcad3c0
[18282.667645] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18282.676660] Call Trace:
[18282.680066] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.688036] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.694253] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.701642] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.708542] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.716508] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.722633] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.730598] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.737291] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.745242] spl_system_task S ffff88081e695e00 0 3149 2 0x00000080
[18282.753827] ffff88082d607e08 0000000000000046 ffff8808257c5e20 ffff88082d607fd8
[18282.762819] ffff88082d607fd8 ffff88082d607fd8 ffff8808257c5e20 ffff88042bcadf60
[18282.771822] ffff88081e695e90 ffffffffa08254b0 ffff88081e695eb0 ffff88081e695e00
[18282.780817] Call Trace:
[18282.784224] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.792194] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.798414] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.805803] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.812704] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.820674] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.826792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.834759] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.841451] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.849410] spl_dynamic_tas S ffff88081e694200 0 3150 2 0x00000080
[18282.857992] ffff8808296f3e08 0000000000000046 ffff8808257c6dd0 ffff8808296f3fd8
[18282.866993] ffff8808296f3fd8 ffff8808296f3fd8 ffff8808257c6dd0 ffff88042bcad5a0
[18282.875998] ffff88081e694290 ffffffffa08254b0 ffff88081e6942b0 ffff88081e694200
[18282.884994] Call Trace:
[18282.888404] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.896381] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18282.902607] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18282.909998] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18282.916891] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18282.924860] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18282.930972] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.938937] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18282.945640] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18282.953598] dbu_evict S ffff88081e695900 0 3155 2 0x00000080
[18282.962180] ffff88082d023e08 0000000000000046 ffff88082d270000 ffff88082d023fd8
[18282.971172] ffff88082d023fd8 ffff88082d023fd8 ffff88082d270000 ffff880826ff6e40
[18282.980173] ffff88081e695990 ffffffffa08254b0 ffff88081e6959b0 ffff88081e695900
[18282.989159] Call Trace:
[18282.992564] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.000549] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.006767] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.014152] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.021054] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.029014] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.035123] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.043079] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.049783] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.057737] arc_prune S ffff88081e694d00 0 3156 2 0x00000080
[18283.066303] ffff880825fc7e08 0000000000000046 ffff88082d270fb0 ffff880825fc7fd8
[18283.075297] ffff880825fc7fd8 ffff880825fc7fd8 ffff88082d270fb0 ffff880826ff6de0
[18283.084285] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.093270] Call Trace:
[18283.096684] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.104654] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.110873] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.118251] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.125145] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.133106] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.139217] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.147181] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.153872] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.161845] arc_prune S ffff88081e694d00 0 3157 2 0x00000080
[18283.170427] ffff8808266e3e08 0000000000000046 ffff88082d271f60 ffff8808266e3fd8
[18283.179430] ffff8808266e3fd8 ffff8808266e3fd8 ffff88082d271f60 ffff880826ff6d80
[18283.188433] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.197416] Call Trace:
[18283.200822] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.208795] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.215026] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.222414] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.229309] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.237270] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.243378] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.251329] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.258021] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.265986] arc_prune S ffff88081e694d00 0 3158 2 0x00000080
[18283.274577] ffff8808266dfe08 0000000000000046 ffff88082d272f10 ffff8808266dffd8
[18283.283570] ffff8808266dffd8 ffff8808266dffd8 ffff88082d272f10 ffff880826ff6d20
[18283.292565] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.301566] Call Trace:
[18283.304968] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.312939] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.319149] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.326523] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.333427] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.341388] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.347506] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.355464] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.362157] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.370121] arc_prune S ffff88081e694d00 0 3159 2 0x00000080
[18283.378692] ffff8808256b3e08 0000000000000046 ffff88082d273ec0 ffff8808256b3fd8
[18283.387686] ffff8808256b3fd8 ffff8808256b3fd8 ffff88082d273ec0 ffff880826ff6cc0
[18283.396696] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.405685] Call Trace:
[18283.409094] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.417071] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.423285] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.430673] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.437571] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.445528] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.451647] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.459613] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.466304] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.474259] arc_prune S ffff88081e694d00 0 3160 2 0x00000080
[18283.482846] ffff88082d267e08 0000000000000046 ffff88082d274e70 ffff88082d267fd8
[18283.491854] ffff88082d267fd8 ffff88082d267fd8 ffff88082d274e70 ffff880826ff6c60
[18283.500860] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.509868] Call Trace:
[18283.513273] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.521236] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.527464] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.534844] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.541750] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.549717] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.555840] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.563804] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.570499] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.578457] arc_prune S ffff88081e694d00 0 3161 2 0x00000080
[18283.587037] ffff88082bddbe08 0000000000000046 ffff88082d275e20 ffff88082bddbfd8
[18283.596040] ffff88082bddbfd8 ffff88082bddbfd8 ffff88082d275e20 ffff880826ff6c00
[18283.605047] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.614048] Call Trace:
[18283.617451] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.625429] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.631653] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.639038] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.645946] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.653909] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.660035] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.667997] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.674697] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.682662] arc_prune S ffff88081e694d00 0 3162 2 0x00000080
[18283.691230] ffff8808253ffe08 0000000000000046 ffff88082d276dd0 ffff8808253fffd8
[18283.700224] ffff8808253fffd8 ffff8808253fffd8 ffff88082d276dd0 ffff880826ff6ba0
[18283.709218] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.718218] Call Trace:
[18283.721634] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.729602] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.735829] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.743203] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.750102] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.758069] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.764172] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.772119] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.778822] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.786781] arc_prune S ffff88081e694d00 0 3163 2 0x00000080
[18283.795344] ffff88082bd63e08 0000000000000046 ffff88083fe10000 ffff88082bd63fd8
[18283.804339] ffff88082bd63fd8 ffff88082bd63fd8 ffff88083fe10000 ffff880826ff6b40
[18283.813339] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.822323] Call Trace:
[18283.825740] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.833719] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.839937] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.847321] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.854222] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.862185] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.868294] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.876248] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.882938] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.890894] arc_prune S ffff88081e694d00 0 3164 2 0x00000080
[18283.899476] ffff88082c277e08 0000000000000046 ffff88083fe10fb0 ffff88082c277fd8
[18283.908467] ffff88082c277fd8 ffff88082c277fd8 ffff88083fe10fb0 ffff880826ff6ae0
[18283.917453] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18283.926446] Call Trace:
[18283.929854] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.937824] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18283.944053] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18283.951439] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18283.958338] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18283.966294] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18283.972397] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.980350] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18283.987051] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18283.995017] arc_prune S ffff88081e694d00 0 3165 2 0x00000080
[18284.003603] ffff880826a3fe08 0000000000000046 ffff88083fe11f60 ffff880826a3ffd8
[18284.012602] ffff880826a3ffd8 ffff880826a3ffd8 ffff88083fe11f60 ffff880826ff6a80
[18284.021602] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.030591] Call Trace:
[18284.034008] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.041986] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.048202] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.055601] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.062501] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.070463] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.076570] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.084547] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.091235] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.099195] arc_prune S ffff88081e694d00 0 3166 2 0x00000080
[18284.107786] ffff880827f6be08 0000000000000046 ffff88083fe12f10 ffff880827f6bfd8
[18284.116802] ffff880827f6bfd8 ffff880827f6bfd8 ffff88083fe12f10 ffff880826ff6a20
[18284.125807] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.134814] Call Trace:
[18284.138224] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.146193] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.152400] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.159791] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.166699] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.174658] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.180775] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.188743] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.195430] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.203381] arc_prune S ffff88081e694d00 0 3167 2 0x00000080
[18284.211961] ffff88082e033e08 0000000000000046 ffff88083fe13ec0 ffff88082e033fd8
[18284.220963] ffff88082e033fd8 ffff88082e033fd8 ffff88083fe13ec0 ffff880826ff69c0
[18284.229968] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.238965] Call Trace:
[18284.242362] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.250327] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.256540] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.263937] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.270839] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.278800] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.284921] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.292876] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.299567] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.307523] arc_prune S ffff88081e694d00 0 3168 2 0x00000080
[18284.316081] ffff880827e4be08 0000000000000046 ffff88083fe14e70 ffff880827e4bfd8
[18284.325082] ffff880827e4bfd8 ffff880827e4bfd8 ffff88083fe14e70 ffff880826ff6960
[18284.334081] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.343075] Call Trace:
[18284.346488] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.354475] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.360705] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.368096] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.374993] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.382960] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.389083] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.397053] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.403746] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.411704] arc_prune S ffff88081e694d00 0 3169 2 0x00000080
[18284.420268] ffff880824bdbe08 0000000000000046 ffff88083fe15e20 ffff880824bdbfd8
[18284.429267] ffff880824bdbfd8 ffff880824bdbfd8 ffff88083fe15e20 ffff880826ff6900
[18284.438259] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.447256] Call Trace:
[18284.450666] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.458638] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.464860] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.472238] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.479129] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.487093] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.493192] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.501153] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.507853] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.515816] arc_prune S ffff88081e694d00 0 3170 2 0x00000080
[18284.524389] ffff8808233efe08 0000000000000046 ffff88083fe16dd0 ffff8808233effd8
[18284.533373] ffff8808233effd8 ffff8808233effd8 ffff88083fe16dd0 ffff880826ff68a0
[18284.542362] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.551361] Call Trace:
[18284.554771] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.562742] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.568959] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.576341] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.583238] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.591195] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.597297] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.605247] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.611948] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.619917] arc_prune S ffff88081e694d00 0 3171 2 0x00000080
[18284.628496] ffff880829707e08 0000000000000046 ffff880825f88000 ffff880829707fd8
[18284.637502] ffff880829707fd8 ffff880829707fd8 ffff880825f88000 ffff880826ff6840
[18284.646509] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.655485] Call Trace:
[18284.658905] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.666886] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.673104] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.680502] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.687400] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.695350] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.701467] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.709415] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.716106] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.724072] arc_prune S ffff88081e694d00 0 3172 2 0x00000080
[18284.732642] ffff880826f3fe08 0000000000000046 ffff880825f88fb0 ffff880826f3ffd8
[18284.741638] ffff880826f3ffd8 ffff880826f3ffd8 ffff880825f88fb0 ffff880826ff67e0
[18284.750637] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.759644] Call Trace:
[18284.763053] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.771035] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.777249] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.784638] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.791545] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.799496] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.805617] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.813576] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.820267] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.828222] arc_prune S ffff88081e694d00 0 3173 2 0x00000080
[18284.836794] ffff880825ef3e08 0000000000000046 ffff880825f89f60 ffff880825ef3fd8
[18284.845799] ffff880825ef3fd8 ffff880825ef3fd8 ffff880825f89f60 ffff880826ff6780
[18284.854802] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.863808] Call Trace:
[18284.867216] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.875175] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.881385] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.888766] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.895668] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.903625] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18284.909737] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.917708] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18284.924392] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18284.932349] arc_prune S ffff88081e694d00 0 3174 2 0x00000080
[18284.940923] ffff88082578fe08 0000000000000046 ffff880825f8af10 ffff88082578ffd8
[18284.949917] ffff88082578ffd8 ffff88082578ffd8 ffff880825f8af10 ffff880826ff6720
[18284.958905] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18284.967898] Call Trace:
[18284.971295] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18284.979260] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18284.985485] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18284.992868] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18284.999772] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.007738] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.013852] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.021804] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.028494] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.036448] arc_prune S ffff88081e694d00 0 3175 2 0x00000080
[18285.045029] ffff880827bcbe08 0000000000000046 ffff880825f8bec0 ffff880827bcbfd8
[18285.054038] ffff880827bcbfd8 ffff880827bcbfd8 ffff880825f8bec0 ffff880826ff66c0
[18285.063035] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.072046] Call Trace:
[18285.075456] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.083427] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.089640] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.097040] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.103958] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.111933] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.118049] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.126009] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.132703] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.140661] arc_prune S ffff88081e694d00 0 3176 2 0x00000080
[18285.149228] ffff8808233dfe08 0000000000000046 ffff880825f8ce70 ffff8808233dffd8
[18285.158219] ffff8808233dffd8 ffff8808233dffd8 ffff880825f8ce70 ffff880826ff6660
[18285.167216] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.176208] Call Trace:
[18285.179630] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.187601] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.193814] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.201205] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.208105] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.216081] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.222190] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.230147] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.236850] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.244813] arc_prune S ffff88081e694d00 0 3177 2 0x00000080
[18285.253380] ffff88082e2f3e08 0000000000000046 ffff880825f8de20 ffff88082e2f3fd8
[18285.262381] ffff88082e2f3fd8 ffff88082e2f3fd8 ffff880825f8de20 ffff880826ff6600
[18285.271383] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.280375] Call Trace:
[18285.283789] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.291760] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.297981] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.305361] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.312257] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.320217] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.326324] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.334280] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.340962] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.348918] arc_prune S ffff88081e694d00 0 3178 2 0x00000080
[18285.357504] ffff88083fb3fe08 0000000000000046 ffff880825f8edd0 ffff88083fb3ffd8
[18285.366494] ffff88083fb3ffd8 ffff88083fb3ffd8 ffff880825f8edd0 ffff880826ff65a0
[18285.375497] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.384485] Call Trace:
[18285.387893] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.395868] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.402092] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.409478] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.416378] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.424339] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.430442] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.438382] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.445082] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.453044] arc_prune S ffff88081e694d00 0 3179 2 0x00000080
[18285.461637] ffff8808256bbe08 0000000000000046 ffff8808263b8000 ffff8808256bbfd8
[18285.470620] ffff8808256bbfd8 ffff8808256bbfd8 ffff8808263b8000 ffff880826ff6540
[18285.479616] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.488621] Call Trace:
[18285.492036] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.500005] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.506218] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.513603] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.520505] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.528463] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.534571] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.542539] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.549238] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.557194] arc_prune S ffff88081e694d00 0 3180 2 0x00000080
[18285.565775] ffff880829bf3e08 0000000000000046 ffff8808263b8fb0 ffff880829bf3fd8
[18285.574771] ffff880829bf3fd8 ffff880829bf3fd8 ffff8808263b8fb0 ffff880826ff64e0
[18285.583776] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.592778] Call Trace:
[18285.596185] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.604150] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.610363] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.617754] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.624654] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.632616] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.638726] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.646681] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.653371] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.661325] arc_prune S ffff88081e694d00 0 3181 2 0x00000080
[18285.669899] ffff88083fdcfe08 0000000000000046 ffff8808263b9f60 ffff88083fdcffd8
[18285.678921] ffff88083fdcffd8 ffff88083fdcffd8 ffff8808263b9f60 ffff880826ff6480
[18285.687922] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.696918] Call Trace:
[18285.700332] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.708302] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.714526] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.721918] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.728816] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.736767] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.742875] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.750841] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.757530] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.765492] arc_prune S ffff88081e694d00 0 3182 2 0x00000080
[18285.774057] ffff880829a63e08 0000000000000046 ffff8808263baf10 ffff880829a63fd8
[18285.783067] ffff880829a63fd8 ffff880829a63fd8 ffff8808263baf10 ffff880826ff6420
[18285.792077] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.801082] Call Trace:
[18285.804494] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.812467] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.818682] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.826076] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.832978] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.840939] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.847061] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.855025] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.861727] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.869692] arc_prune S ffff88081e694d00 0 3183 2 0x00000080
[18285.878261] ffff88083f687e08 0000000000000046 ffff8808263bbec0 ffff88083f687fd8
[18285.887261] ffff88083f687fd8 ffff88083f687fd8 ffff8808263bbec0 ffff880826ff63c0
[18285.896256] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18285.905242] Call Trace:
[18285.908662] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.916635] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18285.922859] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18285.930244] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18285.937149] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18285.945117] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18285.951220] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.959176] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18285.965870] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18285.973836] arc_prune S ffff88081e694d00 0 3184 2 0x00000080
[18285.982400] ffff88083fa7be08 0000000000000046 ffff8808263bce70 ffff88083fa7bfd8
[18285.991391] ffff88083fa7bfd8 ffff88083fa7bfd8 ffff8808263bce70 ffff880826ff6360
[18286.000374] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.009359] Call Trace:
[18286.012772] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.020741] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.026968] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.034346] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.041247] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.049210] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.055319] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.063276] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.069971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.077928] arc_prune S ffff88081e694d00 0 3185 2 0x00000080
[18286.086500] ffff880822ecbe08 0000000000000046 ffff8808263bde20 ffff880822ecbfd8
[18286.095500] ffff880822ecbfd8 ffff880822ecbfd8 ffff8808263bde20 ffff880826ff6300
[18286.104505] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.113501] Call Trace:
[18286.116907] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.124886] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.131102] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.138486] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.145386] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.153345] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.159452] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.167408] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.174114] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.182087] arc_prune S ffff88081e694d00 0 3186 2 0x00000080
[18286.190661] ffff88082bf7be08 0000000000000046 ffff8808263bedd0 ffff88082bf7bfd8
[18286.199664] ffff88082bf7bfd8 ffff88082bf7bfd8 ffff8808263bedd0 ffff880826ff62a0
[18286.208667] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.217652] Call Trace:
[18286.221061] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.229031] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.235245] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.242630] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.249539] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.257515] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.263626] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.271590] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.278273] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.286230] arc_prune S ffff88081e694d00 0 3187 2 0x00000080
[18286.294814] ffff88082daf3e08 0000000000000046 ffff880825230000 ffff88082daf3fd8
[18286.303807] ffff88082daf3fd8 ffff88082daf3fd8 ffff880825230000 ffff880826ff6240
[18286.312810] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.321792] Call Trace:
[18286.325190] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.333161] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.339372] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.346765] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.353665] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.361626] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.367740] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.375703] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.382394] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.390343] arc_prune S ffff88081e694d00 0 3188 2 0x00000080
[18286.398919] ffff880826ab3e08 0000000000000046 ffff880825230fb0 ffff880826ab3fd8
[18286.407901] ffff880826ab3fd8 ffff880826ab3fd8 ffff880825230fb0 ffff880826ff61e0
[18286.416904] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.425907] Call Trace:
[18286.429307] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.437274] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.443482] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.450876] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.457781] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.465740] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.471857] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.479820] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.486511] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.494467] arc_prune S ffff88081e694d00 0 3189 2 0x00000080
[18286.503037] ffff880826fdfe08 0000000000000046 ffff880825231f60 ffff880826fdffd8
[18286.512033] ffff880826fdffd8 ffff880826fdffd8 ffff880825231f60 ffff880826ff6180
[18286.521027] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.530022] Call Trace:
[18286.533430] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.541398] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.547619] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.555016] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.561914] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.569875] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.575984] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.583951] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.590653] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.598608] arc_prune S ffff88081e694d00 0 3190 2 0x00000080
[18286.607192] ffff880827a67e08 0000000000000046 ffff880825232f10 ffff880827a67fd8
[18286.616186] ffff880827a67fd8 ffff880827a67fd8 ffff880825232f10 ffff880826ff6120
[18286.625187] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.634174] Call Trace:
[18286.637589] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.645559] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.651782] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.659172] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.666079] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.674030] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.680141] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.688104] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.694792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.702748] arc_prune S ffff88081e694d00 0 3191 2 0x00000080
[18286.711314] ffff880827a73e08 0000000000000046 ffff880825233ec0 ffff880827a73fd8
[18286.720306] ffff880827a73fd8 ffff880827a73fd8 ffff880825233ec0 ffff880412a55120
[18286.729309] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.738311] Call Trace:
[18286.741729] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.749706] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.755931] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.763318] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.770216] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.778179] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.784289] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.792252] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.798946] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.806895] arc_prune S ffff88081e694d00 0 3192 2 0x00000080
[18286.815454] ffff880827f43e08 0000000000000046 ffff880825234e70 ffff880827f43fd8
[18286.824448] ffff880827f43fd8 ffff880827f43fd8 ffff880825234e70 ffff880412a55180
[18286.833442] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.842438] Call Trace:
[18286.845857] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.853840] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.860079] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.867464] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.874363] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.882320] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.888429] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.896387] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18286.903081] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18286.911054] arc_prune S ffff88081e694d00 0 3193 2 0x00000080
[18286.919630] ffff880827f17e08 0000000000000046 ffff880825235e20 ffff880827f17fd8
[18286.928634] ffff880827f17fd8 ffff880827f17fd8 ffff880825235e20 ffff880412a551e0
[18286.937647] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18286.946638] Call Trace:
[18286.950052] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.958033] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18286.964242] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18286.971628] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18286.978536] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18286.986493] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18286.992604] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.000571] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.007275] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.015227] arc_prune S ffff88081e694d00 0 3194 2 0x00000080
[18287.023815] ffff88082907fe08 0000000000000046 ffff880825236dd0 ffff88082907ffd8
[18287.032821] ffff88082907ffd8 ffff88082907ffd8 ffff880825236dd0 ffff880412a55240
[18287.041839] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.050850] Call Trace:
[18287.054257] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.062224] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.068444] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.075840] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.082739] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.090704] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.096817] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.104776] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.111472] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.119424] arc_prune S ffff88081e694d00 0 3195 2 0x00000080
[18287.128004] ffff88082765be08 0000000000000046 ffff880825f28000 ffff88082765bfd8
[18287.137011] ffff88082765bfd8 ffff88082765bfd8 ffff880825f28000 ffff880412a552a0
[18287.146017] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.155012] Call Trace:
[18287.158417] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.166385] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.172608] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.179996] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.186902] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.194873] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.200995] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.208947] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.215653] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.223621] arc_prune S ffff88081e694d00 0 3196 2 0x00000080
[18287.232195] ffff88082bec3e08 0000000000000046 ffff880825f28fb0 ffff88082bec3fd8
[18287.241195] ffff88082bec3fd8 ffff88082bec3fd8 ffff880825f28fb0 ffff880412a55300
[18287.250191] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.259189] Call Trace:
[18287.262596] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.270568] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.276786] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.284172] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.291080] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.299056] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.305176] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.313142] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.319847] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.327813] arc_prune S ffff88081e694d00 0 3197 2 0x00000080
[18287.336383] ffff880827353e08 0000000000000046 ffff880825f29f60 ffff880827353fd8
[18287.345362] ffff880827353fd8 ffff880827353fd8 ffff880825f29f60 ffff880412a55360
[18287.354361] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.363334] Call Trace:
[18287.366745] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.374722] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.380944] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.388328] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.395230] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.403204] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.409314] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.417269] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.423971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.431934] arc_prune S ffff88081e694d00 0 3198 2 0x00000080
[18287.440502] ffff880826e43e08 0000000000000046 ffff880825f2af10 ffff880826e43fd8
[18287.449481] ffff880826e43fd8 ffff880826e43fd8 ffff880825f2af10 ffff880412a553c0
[18287.458470] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.467448] Call Trace:
[18287.470867] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.478842] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.485060] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.492447] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.499348] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.507308] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.513421] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.521371] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.528073] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.536041] arc_prune S ffff88081e694d00 0 3199 2 0x00000080
[18287.544628] ffff88082bc03e08 0000000000000046 ffff880825f2bec0 ffff88082bc03fd8
[18287.553629] ffff88082bc03fd8 ffff88082bc03fd8 ffff880825f2bec0 ffff880412a55420
[18287.562627] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.571619] Call Trace:
[18287.575032] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.583004] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.589219] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.596605] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.603506] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.611461] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.617580] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.625539] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.632234] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.640197] arc_prune S ffff88081e694d00 0 3200 2 0x00000080
[18287.648793] ffff88081bae3e08 0000000000000046 ffff880825f2ce70 ffff88081bae3fd8
[18287.657811] ffff88081bae3fd8 ffff88081bae3fd8 ffff880825f2ce70 ffff880412a55480
[18287.666818] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.675824] Call Trace:
[18287.679229] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.687216] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.693434] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.700818] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.707722] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.715692] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.721821] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.729789] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.736477] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.744435] arc_prune S ffff88081e694d00 0 3201 2 0x00000080
[18287.753016] ffff88082af17e08 0000000000000046 ffff880825f2de20 ffff88082af17fd8
[18287.762022] ffff88082af17fd8 ffff88082af17fd8 ffff880825f2de20 ffff880412a554e0
[18287.771029] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.780036] Call Trace:
[18287.783445] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.791410] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.797635] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.805021] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.811930] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.819905] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.826014] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.833980] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.840686] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.848655] arc_prune S ffff88081e694d00 0 3202 2 0x00000080
[18287.857227] ffff88082d34be08 0000000000000046 ffff880825f2edd0 ffff88082d34bfd8
[18287.866218] ffff88082d34bfd8 ffff88082d34bfd8 ffff880825f2edd0 ffff880412a55540
[18287.875212] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.884207] Call Trace:
[18287.887620] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.895595] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18287.901820] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18287.909217] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18287.916128] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.924113] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18287.930228] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.938186] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18287.944891] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18287.952857] arc_prune S ffff88081e694d00 0 3203 2 0x00000080
[18287.961434] ffff88082e93fe08 0000000000000046 ffff88082bcc8000 ffff88082e93ffd8
[18287.970424] ffff88082e93ffd8 ffff88082e93ffd8 ffff88082bcc8000 ffff880412a555a0
[18287.979424] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18287.988414] Call Trace:
[18287.991828] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18287.999809] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.006031] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.013420] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.020322] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.028284] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.034398] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.042348] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.049045] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.057006] arc_prune S ffff88081e694d00 0 3204 2 0x00000080
[18288.065582] ffff88082aea3e08 0000000000000046 ffff88082bcc8fb0 ffff88082aea3fd8
[18288.074578] ffff88082aea3fd8 ffff88082aea3fd8 ffff88082bcc8fb0 ffff880412a55600
[18288.083573] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.092575] Call Trace:
[18288.095982] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.103956] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.110180] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.117569] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.124463] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.132426] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.138535] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.146492] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.153193] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.161155] arc_prune S ffff88081e694d00 0 3205 2 0x00000080
[18288.169739] ffff880826303e08 0000000000000046 ffff88082bcc9f60 ffff880826303fd8
[18288.178738] ffff880826303fd8 ffff880826303fd8 ffff88082bcc9f60 ffff880412a55660
[18288.187741] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.196748] Call Trace:
[18288.200176] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.208162] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.214379] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.221773] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.228677] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.236649] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.242752] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.250710] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.257403] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.265366] arc_prune S ffff88081e694d00 0 3206 2 0x00000080
[18288.273962] ffff88083fedbe08 0000000000000046 ffff88082bccaf10 ffff88083fedbfd8
[18288.282963] ffff88083fedbfd8 ffff88083fedbfd8 ffff88082bccaf10 ffff880412a556c0
[18288.291968] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.300964] Call Trace:
[18288.304372] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.312342] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.318555] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.325941] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.332843] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.340806] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.346926] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.354889] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.361583] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.369537] arc_prune S ffff88081e694d00 0 3207 2 0x00000080
[18288.378118] ffff8808277b3e08 0000000000000046 ffff88082bccbec0 ffff8808277b3fd8
[18288.387109] ffff8808277b3fd8 ffff8808277b3fd8 ffff88082bccbec0 ffff880412a55720
[18288.396112] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.405115] Call Trace:
[18288.408526] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.416499] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.422711] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.430113] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.437022] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.444997] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.451116] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.459081] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.465775] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.473736] arc_prune S ffff88081e694d00 0 3208 2 0x00000080
[18288.482322] ffff88082632fe08 0000000000000046 ffff88082bccce70 ffff88082632ffd8
[18288.491322] ffff88082632ffd8 ffff88082632ffd8 ffff88082bccce70 ffff880412a55780
[18288.500314] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.509303] Call Trace:
[18288.512715] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.520697] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.526916] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.534305] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.541213] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.549185] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.555305] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.563264] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.569960] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.577926] arc_prune S ffff88081e694d00 0 3209 2 0x00000080
[18288.586500] ffff88082e4b7e08 0000000000000046 ffff88082bccde20 ffff88082e4b7fd8
[18288.595506] ffff88082e4b7fd8 ffff88082e4b7fd8 ffff88082bccde20 ffff880412a557e0
[18288.604496] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.613491] Call Trace:
[18288.616903] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.624875] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.631096] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.638476] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.645381] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.653343] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.659455] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.667407] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.674101] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.682062] arc_prune S ffff88081e694d00 0 3210 2 0x00000080
[18288.690654] ffff88082dabfe08 0000000000000046 ffff88082bccedd0 ffff88082dabffd8
[18288.699665] ffff88082dabffd8 ffff88082dabffd8 ffff88082bccedd0 ffff880412a55840
[18288.708657] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.717650] Call Trace:
[18288.721073] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.729064] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.735283] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.742670] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.749568] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.757514] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.763618] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.771578] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.778272] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.786231] arc_prune S ffff88081e694d00 0 3211 2 0x00000080
[18288.794815] ffff88082bd73e08 0000000000000046 ffff880827758000 ffff88082bd73fd8
[18288.803818] ffff88082bd73fd8 ffff88082bd73fd8 ffff880827758000 ffff880412a558a0
[18288.812822] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.821817] Call Trace:
[18288.825237] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.833210] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.839426] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.846810] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.853720] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.861690] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.867804] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.875772] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.882466] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.890423] arc_prune S ffff88081e694d00 0 3212 2 0x00000080
[18288.899006] ffff88082c793e08 0000000000000046 ffff880827758fb0 ffff88082c793fd8
[18288.908018] ffff88082c793fd8 ffff88082c793fd8 ffff880827758fb0 ffff880412a55900
[18288.917024] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18288.926051] Call Trace:
[18288.929466] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.937433] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18288.943661] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18288.951040] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18288.957948] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18288.965915] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18288.972032] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.979985] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18288.986675] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18288.994633] arc_prune S ffff88081e694d00 0 3213 2 0x00000080
[18289.003222] ffff880825effe08 0000000000000046 ffff880827759f60 ffff880825efffd8
[18289.012229] ffff880825efffd8 ffff880825efffd8 ffff880827759f60 ffff880412a55960
[18289.021234] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.030239] Call Trace:
[18289.033649] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.041630] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.047847] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.055239] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.062148] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.070119] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.076239] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.084205] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.090910] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.098868] arc_prune S ffff88081e694d00 0 3214 2 0x00000080
[18289.107434] ffff8808273d7e08 0000000000000046 ffff88082775af10 ffff8808273d7fd8
[18289.116423] ffff8808273d7fd8 ffff8808273d7fd8 ffff88082775af10 ffff880412a559c0
[18289.125422] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.134411] Call Trace:
[18289.137818] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.145797] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.152022] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.159409] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.166309] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.174273] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.180375] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.188333] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.195028] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.202993] arc_prune S ffff88081e694d00 0 3215 2 0x00000080
[18289.211569] ffff8808273e7e08 0000000000000046 ffff88082775bec0 ffff8808273e7fd8
[18289.220558] ffff8808273e7fd8 ffff8808273e7fd8 ffff88082775bec0 ffff880412a55a20
[18289.229545] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.238535] Call Trace:
[18289.241959] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.249942] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.256175] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.263563] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.270460] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.278418] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.284529] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.292486] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.299176] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.307151] arc_prune S ffff88081e694d00 0 3216 2 0x00000080
[18289.315746] ffff88082a67be08 0000000000000046 ffff88082775ce70 ffff88082a67bfd8
[18289.324743] ffff88082a67bfd8 ffff88082a67bfd8 ffff88082775ce70 ffff880412a55a80
[18289.333732] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.342726] Call Trace:
[18289.346135] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.354108] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.360324] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.367709] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.374621] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.382579] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.388700] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.396668] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.403363] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.411319] arc_prune S ffff88081e694d00 0 3217 2 0x00000080
[18289.419900] ffff88082a77be08 0000000000000046 ffff88082775de20 ffff88082a77bfd8
[18289.428912] ffff88082a77bfd8 ffff88082a77bfd8 ffff88082775de20 ffff880412a55ae0
[18289.437918] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.446926] Call Trace:
[18289.450335] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.458308] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.464525] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.471921] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.478820] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.486780] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.492900] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.500868] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.507563] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.515524] arc_prune S ffff88081e694d00 0 3218 2 0x00000080
[18289.524115] ffff8808262abe08 0000000000000046 ffff88082775edd0 ffff8808262abfd8
[18289.533122] ffff8808262abfd8 ffff8808262abfd8 ffff88082775edd0 ffff880412a55b40
[18289.542138] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.551135] Call Trace:
[18289.554542] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.562515] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.568743] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.576140] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.583041] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.591001] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.597126] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.605093] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.611788] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.619748] arc_prune S ffff88081e694d00 0 3219 2 0x00000080
[18289.628317] ffff88082a363e08 0000000000000046 ffff8808252c8000 ffff88082a363fd8
[18289.637329] ffff88082a363fd8 ffff88082a363fd8 ffff8808252c8000 ffff880412a55ba0
[18289.646326] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.655329] Call Trace:
[18289.658741] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.666722] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.672954] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.680339] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.687250] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.695223] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.701342] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.709300] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.716002] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.723977] arc_prune S ffff88081e694d00 0 3220 2 0x00000080
[18289.732546] ffff88082e23be08 0000000000000046 ffff8808252c8fb0 ffff88082e23bfd8
[18289.741547] ffff88082e23bfd8 ffff88082e23bfd8 ffff8808252c8fb0 ffff880412a55c00
[18289.750542] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.759538] Call Trace:
[18289.762949] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.770921] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.777138] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.784522] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.791421] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.799389] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.805499] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.813456] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.820157] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.828123] arc_prune S ffff88081e694d00 0 3221 2 0x00000080
[18289.836703] ffff88082e1bfe08 0000000000000046 ffff8808252c9f60 ffff88082e1bffd8
[18289.845700] ffff88082e1bffd8 ffff88082e1bffd8 ffff8808252c9f60 ffff880412a55c60
[18289.854692] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.863682] Call Trace:
[18289.867104] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.875084] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.881303] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18289.888682] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18289.895590] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.903550] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18289.909666] [<ffffffff810bfde3>] ? finish_task_switch+0x53/0x180
[18289.917136] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.925098] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18289.931800] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18289.939757] arc_prune S ffff88081e694d00 0 3222 2 0x00000080
[18289.948316] ffff88082d333e08 0000000000000046 ffff8808252caf10 ffff88082d333fd8
[18289.957318] ffff88082d333fd8 ffff88082d333fd8 ffff8808252caf10 ffff880412a55cc0
[18289.966304] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18289.975292] Call Trace:
[18289.978702] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18289.986674] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18289.992910] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.000293] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.007188] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.015159] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.021275] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.029238] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.035940] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.043907] arc_prune S ffff88081e694d00 0 3223 2 0x00000080
[18290.052465] ffff88082c607e08 0000000000000046 ffff8808252cbec0 ffff88082c607fd8
[18290.061457] ffff88082c607fd8 ffff88082c607fd8 ffff8808252cbec0 ffff880412a55d20
[18290.070448] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.079438] Call Trace:
[18290.082838] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.090804] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.097025] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.104405] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.111298] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.119265] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.125374] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.133324] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.140026] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.147981] arc_prune S ffff88081e694d00 0 3224 2 0x00000080
[18290.156539] ffff88082c233e08 0000000000000046 ffff8808252cce70 ffff88082c233fd8
[18290.165515] ffff88082c233fd8 ffff88082c233fd8 ffff8808252cce70 ffff880412a55d80
[18290.174509] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.183499] Call Trace:
[18290.186905] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.194869] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.201100] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.208489] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.215384] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.223346] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.229457] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.237415] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.244118] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.252079] arc_prune S ffff88081e694d00 0 3225 2 0x00000080
[18290.260652] ffff88083fe0be08 0000000000000046 ffff8808252cde20 ffff88083fe0bfd8
[18290.269653] ffff88083fe0bfd8 ffff88083fe0bfd8 ffff8808252cde20 ffff880412a55de0
[18290.278643] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.287653] Call Trace:
[18290.291059] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.299040] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.305270] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.312670] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.319563] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.327523] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.333633] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.341591] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.348298] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.356262] arc_prune S ffff88081e694d00 0 3226 2 0x00000080
[18290.364840] ffff880827ac7e08 0000000000000046 ffff8808252cedd0 ffff880827ac7fd8
[18290.373846] ffff880827ac7fd8 ffff880827ac7fd8 ffff8808252cedd0 ffff880412a55e40
[18290.382850] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.391847] Call Trace:
[18290.395258] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.403242] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.409457] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.416850] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.423752] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.431712] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.437825] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.445771] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.452465] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.460422] arc_prune S ffff88081e694d00 0 3227 2 0x00000080
[18290.469009] ffff880827acbe08 0000000000000046 ffff880825620000 ffff880827acbfd8
[18290.478008] ffff880827acbfd8 ffff880827acbfd8 ffff880825620000 ffff8800b168d000
[18290.487003] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.495999] Call Trace:
[18290.499404] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.507373] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.513589] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.520986] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.527888] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.535858] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.541977] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.549942] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.556631] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.564581] arc_prune S ffff88081e694d00 0 3228 2 0x00000080
[18290.573160] ffff88082d9f7e08 0000000000000046 ffff880825620fb0 ffff88082d9f7fd8
[18290.582173] ffff88082d9f7fd8 ffff88082d9f7fd8 ffff880825620fb0 ffff8800b168d060
[18290.591170] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.600182] Call Trace:
[18290.603590] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.611556] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.617778] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.625172] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.632080] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.640045] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.646163] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.654127] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.660813] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.668779] arc_prune S ffff88081e694d00 0 3229 2 0x00000080
[18290.677346] ffff8808273c3e08 0000000000000046 ffff880825621f60 ffff8808273c3fd8
[18290.686343] ffff8808273c3fd8 ffff8808273c3fd8 ffff880825621f60 ffff8800b168d0c0
[18290.695334] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.704334] Call Trace:
[18290.707752] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.715734] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.721945] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.729330] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.736243] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.744205] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.750316] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.758276] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.764971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.772929] arc_prune S ffff88081e694d00 0 3230 2 0x00000080
[18290.781495] ffff88082a7f3e08 0000000000000046 ffff880825622f10 ffff88082a7f3fd8
[18290.790488] ffff88082a7f3fd8 ffff88082a7f3fd8 ffff880825622f10 ffff8800b168d120
[18290.799483] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.808482] Call Trace:
[18290.811897] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.819882] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.826105] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.833492] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.840394] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.848355] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.854465] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.862421] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.869124] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.877095] arc_prune S ffff88081e694d00 0 3231 2 0x00000080
[18290.885681] ffff88082977be08 0000000000000046 ffff880825623ec0 ffff88082977bfd8
[18290.894656] ffff88082977bfd8 ffff88082977bfd8 ffff880825623ec0 ffff8800b168d180
[18290.903635] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18290.912629] Call Trace:
[18290.916051] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.924040] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18290.930275] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18290.937661] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18290.944557] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18290.952519] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18290.958630] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.966588] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18290.973288] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18290.981260] arc_prune S ffff88081e694d00 0 3232 2 0x00000080
[18290.989849] ffff88082e533e08 0000000000000046 ffff880825624e70 ffff88082e533fd8
[18290.998858] ffff88082e533fd8 ffff88082e533fd8 ffff880825624e70 ffff8800b168d1e0
[18291.007851] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.016828] Call Trace:
[18291.020235] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.028207] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.034425] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.041813] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.048731] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.056694] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.062808] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.070754] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.077455] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.085414] arc_prune S ffff88081e694d00 0 3233 2 0x00000080
[18291.094000] ffff88082d493e08 0000000000000046 ffff880825625e20 ffff88082d493fd8
[18291.103007] ffff88082d493fd8 ffff88082d493fd8 ffff880825625e20 ffff8800b168d240
[18291.112015] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.121013] Call Trace:
[18291.124420] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.132393] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.138611] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.146000] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.152906] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.160865] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.166977] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.174942] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.181637] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.189582] arc_prune S ffff88081e694d00 0 3234 2 0x00000080
[18291.198176] ffff880826397e08 0000000000000046 ffff880825626dd0 ffff880826397fd8
[18291.207178] ffff880826397fd8 ffff880826397fd8 ffff880825626dd0 ffff8800b168d2a0
[18291.216176] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.225169] Call Trace:
[18291.228569] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.236541] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.242759] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.250146] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.257055] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.265018] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.271124] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.279085] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.285779] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.293747] arc_prune S ffff88081e694d00 0 3235 2 0x00000080
[18291.302327] ffff8808263dbe08 0000000000000046 ffff88082df38000 ffff8808263dbfd8
[18291.311331] ffff8808263dbfd8 ffff8808263dbfd8 ffff88082df38000 ffff8800b168d300
[18291.320337] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.329326] Call Trace:
[18291.332751] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.340718] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.346951] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.354344] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.361260] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.369232] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.375345] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.383305] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.390008] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.397963] arc_prune S ffff88081e694d00 0 3236 2 0x00000080
[18291.406532] ffff880829d4fe08 0000000000000046 ffff88082df38fb0 ffff880829d4ffd8
[18291.415530] ffff880829d4ffd8 ffff880829d4ffd8 ffff88082df38fb0 ffff8800b168d360
[18291.424533] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.433527] Call Trace:
[18291.436939] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.444912] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.451128] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.458513] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.465415] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.473378] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.479489] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.487446] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.494134] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.502098] arc_prune S ffff88081e694d00 0 3237 2 0x00000080
[18291.510672] ffff88083fa2fe08 0000000000000046 ffff88082df39f60 ffff88083fa2ffd8
[18291.519685] ffff88083fa2ffd8 ffff88083fa2ffd8 ffff88082df39f60 ffff8800b168d3c0
[18291.528673] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.537654] Call Trace:
[18291.541073] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.549052] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.555275] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.562655] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.569556] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.577512] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.583614] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.591572] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.598276] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.606244] arc_prune S ffff88081e694d00 0 3238 2 0x00000080
[18291.614830] ffff880824b4be08 0000000000000046 ffff88082df3af10 ffff880824b4bfd8
[18291.623825] ffff880824b4bfd8 ffff880824b4bfd8 ffff88082df3af10 ffff8800b168d420
[18291.632813] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.641812] Call Trace:
[18291.645225] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.653198] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.659427] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.666822] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.673725] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.681688] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.687803] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.695762] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.702457] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.710402] arc_prune S ffff88081e694d00 0 3239 2 0x00000080
[18291.718984] ffff88081d603e08 0000000000000046 ffff88082df3bec0 ffff88081d603fd8
[18291.727987] ffff88081d603fd8 ffff88081d603fd8 ffff88082df3bec0 ffff8800b168d480
[18291.736998] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.746020] Call Trace:
[18291.749428] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.757401] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.763605] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.771001] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.777903] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.785865] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.791989] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.799950] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.806645] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.814605] arc_prune S ffff88081e694d00 0 3240 2 0x00000080
[18291.823184] ffff88082362be08 0000000000000046 ffff88082df3ce70 ffff88082362bfd8
[18291.832195] ffff88082362bfd8 ffff88082362bfd8 ffff88082df3ce70 ffff8800b168d4e0
[18291.841199] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.850217] Call Trace:
[18291.853629] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.861593] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.867820] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.875214] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.882111] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.890074] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18291.896194] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.904150] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18291.910839] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18291.918794] arc_prune S ffff88081e694d00 0 3241 2 0x00000080
[18291.927374] ffff88082abfbe08 0000000000000046 ffff88082df3de20 ffff88082abfbfd8
[18291.936374] ffff88082abfbfd8 ffff88082abfbfd8 ffff88082df3de20 ffff8800b168d540
[18291.945367] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18291.954366] Call Trace:
[18291.957780] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.965759] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18291.971976] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18291.979374] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18291.986284] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18291.994249] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.000350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.008321] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.015024] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.022982] arc_prune S ffff88081e694d00 0 3242 2 0x00000080
[18292.031561] ffff880825ee3e08 0000000000000046 ffff88082df3edd0 ffff880825ee3fd8
[18292.040553] ffff880825ee3fd8 ffff880825ee3fd8 ffff88082df3edd0 ffff8800b168d5a0
[18292.049555] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.058548] Call Trace:
[18292.061966] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.069948] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.076167] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.083553] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.090449] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.098413] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.104515] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.112473] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.119171] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.127144] arc_prune S ffff88081e694d00 0 3243 2 0x00000080
[18292.135736] ffff880826327e08 0000000000000046 ffff880826740000 ffff880826327fd8
[18292.144735] ffff880826327fd8 ffff880826327fd8 ffff880826740000 ffff8800b168d600
[18292.153733] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.162717] Call Trace:
[18292.166126] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.174102] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.180321] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.187719] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.194620] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.202583] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.208696] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.216649] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.223350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.231310] arc_prune S ffff88081e694d00 0 3244 2 0x00000080
[18292.239901] ffff88082568be08 0000000000000046 ffff880826740fb0 ffff88082568bfd8
[18292.248886] ffff88082568bfd8 ffff88082568bfd8 ffff880826740fb0 ffff8800b168d660
[18292.257888] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.266881] Call Trace:
[18292.270301] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.278283] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.284498] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.291885] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.298788] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.306753] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.312874] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.320848] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.327544] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.335508] arc_prune S ffff88081e694d00 0 3245 2 0x00000080
[18292.344096] ffff88083fea7e08 0000000000000046 ffff880826741f60 ffff88083fea7fd8
[18292.353103] ffff88083fea7fd8 ffff88083fea7fd8 ffff880826741f60 ffff8800b168d6c0
[18292.362112] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.371113] Call Trace:
[18292.374520] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.382492] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.388712] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.396098] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.403006] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.410976] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.417088] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.425055] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.431748] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.439711] arc_prune S ffff88081e694d00 0 3246 2 0x00000080
[18292.448300] ffff880829d2be08 0000000000000046 ffff880826742f10 ffff880829d2bfd8
[18292.457311] ffff880829d2bfd8 ffff880829d2bfd8 ffff880826742f10 ffff8800b168d720
[18292.466320] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.475322] Call Trace:
[18292.478738] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.486707] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.492928] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.500325] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.507232] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.515205] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.521329] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.529303] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.536003] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.543959] arc_prune S ffff88081e694d00 0 3247 2 0x00000080
[18292.552520] ffff8808276a7e08 0000000000000046 ffff880826743ec0 ffff8808276a7fd8
[18292.561517] ffff8808276a7fd8 ffff8808276a7fd8 ffff880826743ec0 ffff8800b168d780
[18292.570514] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.579499] Call Trace:
[18292.582918] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.590898] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.597122] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.604509] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.611405] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.619372] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.625484] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.633446] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.640138] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.648106] arc_prune S ffff88081e694d00 0 3248 2 0x00000080
[18292.656676] ffff880829767e08 0000000000000046 ffff880826744e70 ffff880829767fd8
[18292.665663] ffff880829767fd8 ffff880829767fd8 ffff880826744e70 ffff8800b168d7e0
[18292.674660] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.683653] Call Trace:
[18292.687061] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.695034] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.701252] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.708640] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.715540] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.723503] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.729607] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.737559] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.744262] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.752222] arc_prune S ffff88081e694d00 0 3249 2 0x00000080
[18292.760812] ffff88082979be08 0000000000000046 ffff880826745e20 ffff88082979bfd8
[18292.769796] ffff88082979bfd8 ffff88082979bfd8 ffff880826745e20 ffff8800b168d840
[18292.778805] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.787801] Call Trace:
[18292.791208] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.799194] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.805411] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.812805] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.819701] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.827662] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.833772] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.841742] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.848440] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.856395] arc_prune S ffff88081e694d00 0 3250 2 0x00000080
[18292.864968] ffff88082c707e08 0000000000000046 ffff880826746dd0 ffff88082c707fd8
[18292.873976] ffff88082c707fd8 ffff88082c707fd8 ffff880826746dd0 ffff8800b168d8a0
[18292.882968] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.891952] Call Trace:
[18292.895380] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.903358] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18292.909574] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18292.916962] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18292.923872] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18292.931834] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18292.937952] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.945918] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18292.952611] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18292.960569] arc_prune S ffff88081e694d00 0 3251 2 0x00000080
[18292.969159] ffff8808253bfe08 0000000000000046 ffff88082ab60000 ffff8808253bffd8
[18292.978167] ffff8808253bffd8 ffff8808253bffd8 ffff88082ab60000 ffff8800b168d900
[18292.987155] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18292.996139] Call Trace:
[18292.999545] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.007516] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.013735] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.021124] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.028034] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.036003] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.042123] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.050081] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.056782] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.064758] arc_prune S ffff88081e694d00 0 3252 2 0x00000080
[18293.073345] ffff88082bef7e08 0000000000000046 ffff88082ab60fb0 ffff88082bef7fd8
[18293.082349] ffff88082bef7fd8 ffff88082bef7fd8 ffff88082ab60fb0 ffff8800b168d960
[18293.091356] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.100356] Call Trace:
[18293.103779] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.111759] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.117995] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.125393] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.132311] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.140277] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.146394] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.154358] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.161057] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.169024] arc_prune S ffff88081e694d00 0 3253 2 0x00000080
[18293.177582] ffff880829f8fe08 0000000000000046 ffff88082ab61f60 ffff880829f8ffd8
[18293.186577] ffff880829f8ffd8 ffff880829f8ffd8 ffff88082ab61f60 ffff8800b168d9c0
[18293.195565] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.204554] Call Trace:
[18293.207967] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.215940] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.222157] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.229548] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.236443] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.244400] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.250517] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.258472] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.265168] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.273134] arc_prune S ffff88081e694d00 0 3254 2 0x00000080
[18293.281700] ffff88082d6afe08 0000000000000046 ffff88082ab62f10 ffff88082d6affd8
[18293.290687] ffff88082d6affd8 ffff88082d6affd8 ffff88082ab62f10 ffff8800b168da20
[18293.299679] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.308669] Call Trace:
[18293.312071] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.320059] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.326286] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.333673] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.340577] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.348540] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.354650] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.362609] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.369313] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.377278] arc_prune S ffff88081e694d00 0 3255 2 0x00000080
[18293.385865] ffff88082d8ffe08 0000000000000046 ffff88082ab63ec0 ffff88082d8fffd8
[18293.394862] ffff88082d8fffd8 ffff88082d8fffd8 ffff88082ab63ec0 ffff8800b168da80
[18293.403862] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.412859] Call Trace:
[18293.416264] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.424239] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.430455] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.437851] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.444756] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.452718] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.458828] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.466792] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.473484] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.481440] arc_prune S ffff88081e694d00 0 3256 2 0x00000080
[18293.490040] ffff88082dc3fe08 0000000000000046 ffff88082ab64e70 ffff88082dc3ffd8
[18293.499041] ffff88082dc3ffd8 ffff88082dc3ffd8 ffff88082ab64e70 ffff8800b168dae0
[18293.508037] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.517059] Call Trace:
[18293.520471] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.528443] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.534661] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.542055] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.548958] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.556916] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.563022] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.570974] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.577662] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.585614] arc_prune S ffff88081e694d00 0 3257 2 0x00000080
[18293.594187] ffff88083fecfe08 0000000000000046 ffff88082ab65e20 ffff88083fecffd8
[18293.603174] ffff88083fecffd8 ffff88083fecffd8 ffff88082ab65e20 ffff8800b168db40
[18293.612166] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.621156] Call Trace:
[18293.624557] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.632523] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.638734] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.646129] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.653038] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.660998] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.667120] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.675085] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.681765] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.689724] arc_prune S ffff88081e694d00 0 3258 2 0x00000080
[18293.698310] ffff88082e4bbe08 0000000000000046 ffff88082ab66dd0 ffff88082e4bbfd8
[18293.707314] ffff88082e4bbfd8 ffff88082e4bbfd8 ffff88082ab66dd0 ffff8800b168dba0
[18293.716320] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.725319] Call Trace:
[18293.728730] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.736701] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.742918] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.750315] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.757219] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.765184] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.771296] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.779259] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.785962] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.793930] arc_prune S ffff88081e694d00 0 3259 2 0x00000080
[18293.802509] ffff88082dd37e08 0000000000000046 ffff880826be0000 ffff88082dd37fd8
[18293.811498] ffff88082dd37fd8 ffff88082dd37fd8 ffff880826be0000 ffff8800b168dc00
[18293.820484] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.829481] Call Trace:
[18293.832903] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.840875] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.847103] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.854483] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.861406] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.869385] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.875489] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.883449] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.890158] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.898123] arc_prune S ffff88081e694d00 0 3260 2 0x00000080
[18293.906694] ffff880829ac3e08 0000000000000046 ffff880826be0fb0 ffff880829ac3fd8
[18293.915684] ffff880829ac3fd8 ffff880829ac3fd8 ffff880826be0fb0 ffff8800b168dc60
[18293.924671] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18293.933662] Call Trace:
[18293.937085] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.945068] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18293.951295] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18293.958680] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18293.965583] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18293.973538] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18293.979633] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18293.987588] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18293.994297] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.002267] arc_prune S ffff88081e694d00 0 3261 2 0x00000080
[18294.010840] ffff88082ae6be08 0000000000000046 ffff880826be1f60 ffff88082ae6bfd8
[18294.019848] ffff88082ae6bfd8 ffff88082ae6bfd8 ffff880826be1f60 ffff8800b168dcc0
[18294.028842] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.037849] Call Trace:
[18294.041267] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.049250] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.055460] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.062838] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.069736] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.077698] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.083822] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.091787] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.098480] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.106442] arc_prune S ffff88081e694d00 0 3262 2 0x00000080
[18294.115010] ffff88082a6afe08 0000000000000046 ffff880826be2f10 ffff88082a6affd8
[18294.124011] ffff88082a6affd8 ffff88082a6affd8 ffff880826be2f10 ffff8800b168dd20
[18294.133015] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.142014] Call Trace:
[18294.145431] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.153409] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.159624] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.167010] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.173912] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.181875] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.187987] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.195942] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.202620] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.210573] arc_prune S ffff88081e694d00 0 3263 2 0x00000080
[18294.219155] ffff88082e33be08 0000000000000046 ffff880826be3ec0 ffff88082e33bfd8
[18294.228153] ffff88082e33bfd8 ffff88082e33bfd8 ffff880826be3ec0 ffff8800b168dd80
[18294.237162] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.246164] Call Trace:
[18294.249568] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.257538] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.263746] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.271125] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.278028] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.285992] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.292106] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.300066] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.306753] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.314705] arc_prune S ffff88081e694d00 0 3264 2 0x00000080
[18294.323275] ffff88082e7b7e08 0000000000000046 ffff880826be4e70 ffff88082e7b7fd8
[18294.332275] ffff88082e7b7fd8 ffff88082e7b7fd8 ffff880826be4e70 ffff8800b168dde0
[18294.341268] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.350273] Call Trace:
[18294.353669] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.361633] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.367853] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.375235] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.382139] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.390105] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.396210] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.404170] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.410859] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.418810] arc_prune S ffff88081e694d00 0 3265 2 0x00000080
[18294.427379] ffff880829c1fe08 0000000000000046 ffff880826be5e20 ffff880829c1ffd8
[18294.436372] ffff880829c1ffd8 ffff880829c1ffd8 ffff880826be5e20 ffff8800b168de40
[18294.445359] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.454340] Call Trace:
[18294.457737] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.465703] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.471922] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.479303] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.486207] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.494171] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.500284] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.508235] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.514924] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.522878] arc_prune S ffff88081e694d00 0 3266 2 0x00000080
[18294.531435] ffff88082ddfbe08 0000000000000046 ffff880826be6dd0 ffff88082ddfbfd8
[18294.540420] ffff88082ddfbfd8 ffff88082ddfbfd8 ffff880826be6dd0 ffff8800b168dea0
[18294.549408] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.558400] Call Trace:
[18294.561810] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.569785] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.576006] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.583389] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.590294] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.598261] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.604376] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.612337] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.619026] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.626979] arc_prune S ffff88081e694d00 0 3267 2 0x00000080
[18294.635553] ffff88082be07e08 0000000000000046 ffff88082e9f0000 ffff88082be07fd8
[18294.644538] ffff88082be07fd8 ffff88082be07fd8 ffff88082e9f0000 ffff8800b168df00
[18294.653521] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.662507] Call Trace:
[18294.665909] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.673876] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.680090] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.687472] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.694371] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.702329] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.708436] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.716391] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.723090] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.731058] arc_prune S ffff88081e694d00 0 3268 2 0x00000080
[18294.739622] ffff88082dab3e08 0000000000000046 ffff88082e9f0fb0 ffff88082dab3fd8
[18294.748622] ffff88082dab3fd8 ffff88082dab3fd8 ffff88082e9f0fb0 ffff8800b168df60
[18294.757619] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.766620] Call Trace:
[18294.770030] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.778001] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.784226] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.791615] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.798519] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.806481] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.812590] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.820548] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.827251] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.835217] arc_prune S ffff88081e694d00 0 3269 2 0x00000080
[18294.843783] ffff88083fbffe08 0000000000000046 ffff88082e9f1f60 ffff88083fbfffd8
[18294.852798] ffff88083fbfffd8 ffff88083fbfffd8 ffff88082e9f1f60 ffff8800b1698000
[18294.861789] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.870774] Call Trace:
[18294.874178] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.882150] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.888380] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.895764] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18294.902657] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.910619] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18294.916730] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.924689] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18294.931384] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18294.939338] arc_prune S ffff88081e694d00 0 3270 2 0x00000080
[18294.947922] ffff88082a68be08 0000000000000046 ffff88082e9f2f10 ffff88082a68bfd8
[18294.956920] ffff88082a68bfd8 ffff88082a68bfd8 ffff88082e9f2f10 ffff8800b1698060
[18294.965915] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18294.974909] Call Trace:
[18294.978333] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18294.986306] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18294.992517] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18294.999900] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.006802] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.014764] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.020884] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.028851] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.035548] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.043508] arc_prune S ffff88081e694d00 0 3271 2 0x00000080
[18295.052092] ffff880829a73e08 0000000000000046 ffff88082e9f3ec0 ffff880829a73fd8
[18295.061094] ffff880829a73fd8 ffff880829a73fd8 ffff88082e9f3ec0 ffff8800b16980c0
[18295.070104] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.079110] Call Trace:
[18295.082513] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.090491] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.096702] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.104100] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.111005] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.118973] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.125088] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.133054] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.139749] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.147705] arc_prune S ffff88081e694d00 0 3272 2 0x00000080
[18295.156295] ffff88082dc37e08 0000000000000046 ffff88082e9f4e70 ffff88082dc37fd8
[18295.165301] ffff88082dc37fd8 ffff88082dc37fd8 ffff88082e9f4e70 ffff8800b1698120
[18295.174300] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.183288] Call Trace:
[18295.186696] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.194667] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.200886] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.208271] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.215178] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.223138] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.229263] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.237223] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.243935] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.251904] arc_prune S ffff88081e694d00 0 3273 2 0x00000080
[18295.260483] ffff88082e83fe08 0000000000000046 ffff88082e9f5e20 ffff88082e83ffd8
[18295.269478] ffff88082e83ffd8 ffff88082e83ffd8 ffff88082e9f5e20 ffff8800b1698180
[18295.278467] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.287452] Call Trace:
[18295.290861] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.298841] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.305059] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.312451] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.319362] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.327327] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.333453] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.341425] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.348120] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.356077] arc_prune S ffff88081e694d00 0 3274 2 0x00000080
[18295.364644] ffff88082e0f7e08 0000000000000046 ffff88082e9f6dd0 ffff88082e0f7fd8
[18295.373636] ffff88082e0f7fd8 ffff88082e0f7fd8 ffff88082e9f6dd0 ffff8800b16981e0
[18295.382618] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.391614] Call Trace:
[18295.395025] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.402995] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.409211] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.416593] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.423487] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.431457] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.437569] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.445531] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.452227] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.460192] arc_prune S ffff88081e694d00 0 3275 2 0x00000080
[18295.468763] ffff88082bdcfe08 0000000000000046 ffff88083f7a8000 ffff88082bdcffd8
[18295.477750] ffff88082bdcffd8 ffff88082bdcffd8 ffff88083f7a8000 ffff8800b1698240
[18295.486745] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.495737] Call Trace:
[18295.499149] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.507121] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.513344] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.520728] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.527628] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.535587] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.541691] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.549634] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.556335] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.564307] arc_prune S ffff88081e694d00 0 3276 2 0x00000080
[18295.572895] ffff88082c6abe08 0000000000000046 ffff88083f7a8fb0 ffff88082c6abfd8
[18295.581899] ffff88082c6abfd8 ffff88082c6abfd8 ffff88083f7a8fb0 ffff8800b16982a0
[18295.590897] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.599893] Call Trace:
[18295.603307] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.611278] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.617487] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.624873] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.631780] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.639743] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.645856] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.653814] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.660508] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.668471] arc_prune S ffff88081e694d00 0 3277 2 0x00000080
[18295.677047] ffff88082d103e08 0000000000000046 ffff88083f7a9f60 ffff88082d103fd8
[18295.686047] ffff88082d103fd8 ffff88082d103fd8 ffff88083f7a9f60 ffff8800b1698300
[18295.695037] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.704021] Call Trace:
[18295.707439] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.715425] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.721640] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.729033] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.735937] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.743898] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.750015] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.757973] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.764660] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.772613] arc_prune S ffff88081e694d00 0 3278 2 0x00000080
[18295.781199] ffff88082dcf7e08 0000000000000046 ffff88083f7aaf10 ffff88082dcf7fd8
[18295.790203] ffff88082dcf7fd8 ffff88082dcf7fd8 ffff88083f7aaf10 ffff8800b1698360
[18295.799192] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.808208] Call Trace:
[18295.811612] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.819582] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.825797] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.833194] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.840095] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.848067] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.854187] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.862152] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.868846] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.876803] arc_prune S ffff88081e694d00 0 3279 2 0x00000080
[18295.885387] ffff88082e03fe08 0000000000000046 ffff88083f7abec0 ffff88082e03ffd8
[18295.894386] ffff88082e03ffd8 ffff88082e03ffd8 ffff88083f7abec0 ffff8800b16983c0
[18295.903395] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18295.912394] Call Trace:
[18295.915799] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.923772] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18295.929998] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18295.937391] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18295.944294] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18295.952258] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18295.958370] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.966327] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18295.973021] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18295.980979] arc_prune S ffff88081e694d00 0 3280 2 0x00000080
[18295.989566] ffff88082c977e08 0000000000000046 ffff88083f7ace70 ffff88082c977fd8
[18295.998556] ffff88082c977fd8 ffff88082c977fd8 ffff88083f7ace70 ffff8800b1698420
[18296.007556] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.016553] Call Trace:
[18296.019958] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.027931] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.034137] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.041525] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.048435] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.056410] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.062514] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.070478] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.077181] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.085133] arc_prune S ffff88081e694d00 0 3281 2 0x00000080
[18296.093700] ffff88082a7ebe08 0000000000000046 ffff88083f7ade20 ffff88082a7ebfd8
[18296.102696] ffff88082a7ebfd8 ffff88082a7ebfd8 ffff88083f7ade20 ffff8800b1698480
[18296.111699] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.120699] Call Trace:
[18296.124112] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.132085] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.138305] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.145689] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.152588] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.160547] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.166655] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.174618] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.181310] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.189273] arc_prune S ffff88081e694d00 0 3282 2 0x00000080
[18296.197851] ffff88082dff7e08 0000000000000046 ffff88083f7aedd0 ffff88082dff7fd8
[18296.206837] ffff88082dff7fd8 ffff88082dff7fd8 ffff88083f7aedd0 ffff8800b16984e0
[18296.215839] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.224823] Call Trace:
[18296.228244] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.236224] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.242446] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.249824] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.256732] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.264692] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.270803] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.278760] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.285467] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.293440] arc_prune S ffff88081e694d00 0 3283 2 0x00000080
[18296.302021] ffff88082e1ffe08 0000000000000046 ffff880826690000 ffff88082e1fffd8
[18296.311005] ffff88082e1fffd8 ffff88082e1fffd8 ffff880826690000 ffff8800b1698540
[18296.320002] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.329008] Call Trace:
[18296.332423] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.340402] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.346616] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.353997] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.360913] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.368883] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.374991] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.382954] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.389647] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.397605] arc_prune S ffff88081e694d00 0 3284 2 0x00000080
[18296.406191] ffff88082e777e08 0000000000000046 ffff880826690fb0 ffff88082e777fd8
[18296.415196] ffff88082e777fd8 ffff88082e777fd8 ffff880826690fb0 ffff8800b16985a0
[18296.424195] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.433187] Call Trace:
[18296.436593] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.444556] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.450766] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.458150] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.465050] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.473030] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.479152] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.487095] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.493786] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.501735] arc_prune S ffff88081e694d00 0 3285 2 0x00000080
[18296.510328] ffff8808297dfe08 0000000000000046 ffff880826691f60 ffff8808297dffd8
[18296.519338] ffff8808297dffd8 ffff8808297dffd8 ffff880826691f60 ffff8800b1698600
[18296.528339] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.537348] Call Trace:
[18296.540762] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.548730] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.554946] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.562327] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.569234] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.577202] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.583313] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.591263] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.597963] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.605921] arc_prune S ffff88081e694d00 0 3286 2 0x00000080
[18296.614509] ffff88082a6d7e08 0000000000000046 ffff880826692f10 ffff88082a6d7fd8
[18296.623516] ffff88082a6d7fd8 ffff88082a6d7fd8 ffff880826692f10 ffff8800b1698660
[18296.632507] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.641504] Call Trace:
[18296.644910] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.652892] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.659113] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.666507] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.673419] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.681378] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.687491] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.695454] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.702146] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.710104] arc_prune S ffff88081e694d00 0 3287 2 0x00000080
[18296.718670] ffff880829bc7e08 0000000000000046 ffff880826693ec0 ffff880829bc7fd8
[18296.727670] ffff880829bc7fd8 ffff880829bc7fd8 ffff880826693ec0 ffff8800b16986c0
[18296.736669] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.745673] Call Trace:
[18296.749084] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.757060] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.763283] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.770661] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.777558] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.785522] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.791614] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.799573] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.806269] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.814227] arc_prune S ffff88081e694d00 0 3288 2 0x00000080
[18296.822792] ffff880829fb3e08 0000000000000046 ffff880826694e70 ffff880829fb3fd8
[18296.831787] ffff880829fb3fd8 ffff880829fb3fd8 ffff880826694e70 ffff8800b1698720
[18296.840787] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.849771] Call Trace:
[18296.853172] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.861142] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.867358] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.874741] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.881638] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.889590] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.895693] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.903648] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18296.910338] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18296.918295] arc_prune S ffff88081e694d00 0 3289 2 0x00000080
[18296.926884] ffff88082afe3e08 0000000000000046 ffff880826695e20 ffff88082afe3fd8
[18296.935889] ffff88082afe3fd8 ffff88082afe3fd8 ffff880826695e20 ffff8800b1698780
[18296.944881] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18296.953871] Call Trace:
[18296.957273] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.965243] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18296.971463] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18296.978842] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18296.985741] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18296.993700] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18296.999809] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.007765] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18297.014464] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.022428] arc_prune S ffff88081e694d00 0 3290 2 0x00000080
[18297.031010] ffff880829733e08 0000000000000046 ffff880826696dd0 ffff880829733fd8
[18297.040006] ffff880829733fd8 ffff880829733fd8 ffff880826696dd0 ffff8800b16987e0
[18297.048999] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18297.057985] Call Trace:
[18297.061388] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.069357] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18297.075570] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18297.082956] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18297.089847] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.097806] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18297.103914] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.111876] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18297.118570] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.126537] arc_prune S ffff88081e694d00 0 3291 2 0x00000080
[18297.135122] ffff88082d33fe08 0000000000000046 ffff88082d158000 ffff88082d33ffd8
[18297.144134] ffff88082d33ffd8 ffff88082d33ffd8 ffff88082d158000 ffff8800b1698840
[18297.153137] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18297.162139] Call Trace:
[18297.165550] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.173526] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18297.179741] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18297.187127] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18297.194038] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.202009] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18297.208120] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.216078] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18297.222771] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.230728] arc_prune S ffff88081e694d00 0 3292 2 0x00000080
[18297.239314] ffff88082de3be08 0000000000000046 ffff88082d158fb0 ffff88082de3bfd8
[18297.248322] ffff88082de3bfd8 ffff88082de3bfd8 ffff88082d158fb0 ffff8800b16988a0
[18297.257330] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18297.266339] Call Trace:
[18297.269747] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.277719] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18297.283935] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18297.291335] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18297.298240] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.306209] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18297.312323] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.320289] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18297.326984] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.334949] arc_prune S ffff88081e694d00 0 3293 2 0x00000080
[18297.343529] ffff880829f9fe08 0000000000000046 ffff88082d159f60 ffff880829f9ffd8
[18297.352538] ffff880829f9ffd8 ffff880829f9ffd8 ffff88082d159f60 ffff8800b1698900
[18297.361541] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18297.370532] Call Trace:
[18297.373949] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.381925] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18297.388148] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18297.395547] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18297.402451] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.410414] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18297.416540] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.424513] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18297.431205] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.439170] arc_prune S ffff88081e694d00 0 3294 2 0x00000080
[18297.447728] ffff88082a35fe08 0000000000000046 ffff88082d15af10 ffff88082a35ffd8
[18297.456731] ffff88082a35ffd8 ffff88082a35ffd8 ffff88082d15af10 ffff8800b1698960
[18297.465718] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18297.474717] Call Trace:
[18297.478140] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.479869] NMI watchdog: BUG: soft lockup - CPU#24 stuck for 22s! [OI_scrub:5551]
Message from[18297.479896] Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache osp(OE) mdd(OE) lod(OE) mdt(OE) lfsck(OE) mgs(OE) mgc(OE) osd_ldiskfs(OE) ldiskfs(OE) lquota(OE) fid(OE) fld(OE) ko2iblnd(OE) ptlrpc(OE) obdclass(OE) lnet(OE) libcfs(OE) zfs(POE) zunicode(POE) zavl(POE) zcommon(POE) znvpair(POE) spl(OE) zlib_deflate 8021q garp mrp stp llc rpcrdma ib_isert iscsi_target_mod ib_iser libiscsi scsi_transport_iscsi ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx4_ib ib_core intel_powerclamp coretemp intel_rapl iosf_mbi kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd dm_round_robin ipmi_ssif sb_edac ipmi_devintf ntb sg iTCO_wdt ioatdma shpchp edac_core mei_me iTCO_vendor_support syslogd@soak-8 at May 9 23:14:09 ...
kernel mei lpc_ich ipmi_si pcspkr i2c_i801 ipmi_msghandler wmi nfsd dm_multipath dm_mod nfs_acl lockd grace auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mlx4_en mgag200 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops isci igb ttm ahci crct10dif_pclmul crct10dif_common ptp libsas crc32c_intel libahci pps_core drm mlx4_core mpt2sas libata dca raid_class i2c_algo_bit scsi_transport_sas devlink i2c_core fjes
:NMI watchdog: BUG: soft lockup - CPU#24 stuck f[18297.479912] CPU: 24 PID: 5551 Comm: OI_scrub Tainted: P OEL ------------ 3.10.0-514.16.1.el7_lustre.x86_64 #1
or 22s! [OI_scru[18297.479913] Hardware name: Intel Corporation SandyBridge Platform/To be filled by O.E.M., BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013
b:5551]
[18297.479913] task: ffff88083fde6dd0 ti: ffff880703600000 task.ti: ffff880703600000
[18297.479933] RIP: 0010:[<ffffffffa121cfb9>] [<ffffffffa121cfb9>] osd_inode_iteration+0x269/0xcc0 [osd_ldiskfs]
[18297.479934] RSP: 0018:ffff880703603d18 EFLAGS: 00000202
[18297.479934] RAX: 00000000000072ce RBX: 0000000000000001 RCX: ffff88042bd78800
[18297.479935] RDX: 00000000000072ce RSI: ffff880419dc3a00 RDI: ffff88042bd79800
[18297.479936] RBP: ffff880703603df0 R08: ffff880703603d57 R09: 0000000000000004
[18297.479945] R10: 000000009f8f2001 R11: ffffea00027e3c80 R12: 000000009f8f2001
[18297.479945] R13: ffffea00027e3c80 R14: 000000009f8f2001 R15: ffff880703603d08
[18297.479946] FS: 0000000000000000(0000) GS:ffff88082da00000(0000) knlGS:0000000000000000
[18297.479947] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[18297.479948] CR2: 00007f465d19147e CR3: 00000000019be000 CR4: 00000000000407e0
[18297.479948] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[18297.479949] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[18297.479949] Stack:
[18297.479950] ffffffffa121b990 ffffffffa1217a20 ffff8800b2a36000 00000000810d354f
[18297.479951] ffff8803162f6000 ffff8800b2a37468 0000000020000000 010000000000000c
[18297.479952] 0000000000000000 0000000000000000 ffff8800b2a36000 0000000000000000
[18297.479952] Call Trace:
[18297.479960] [<ffffffffa121b990>] ? osd_ios_ROOT_scan+0x300/0x300 [osd_ldiskfs]
[18297.479966] [<ffffffffa1217a20>] ? osd_preload_next+0xb0/0xb0 [osd_ldiskfs]
[18297.479972] [<ffffffffa121e370>] osd_scrub_main+0x960/0xf30 [osd_ldiskfs]
[18297.479974] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18297.479980] [<ffffffffa121da10>] ? osd_inode_iteration+0xcc0/0xcc0 [osd_ldiskfs]
[18297.479982] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18297.479984] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.479985] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18297.479986] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.479996] Code: 88 e8 dc 2d f5 ff 48 85 c0 48 89 45 90 0f 84 b3 08 00 00 44 8b 7d a0 48 8b 7d 88 48 89 de 48 c7 45 a8 00 00 00 00 e8 77 5b f4 ff <42> 8d 14 38 48 8b 45 88 48 8b 80 a8 03 00 00 48 3b 50 20 0f 83
[18297.906787] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18297.913315] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18297.921006] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18297.928202] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18297.936468] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18297.942876] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.951102] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18297.958062] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18297.966267] arc_prune S ffff88081e694d00 0 3295 2 0x00000080
[18297.975066] ffff88082c673e08 0000000000000046 ffff88082d15bec0 ffff88082c673fd8
[18297.984265] ffff88082c673fd8 ffff88082c673fd8 ffff88082d15bec0 ffff8800b16989c0
[18297.993442] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.002596] Call Trace:
[18298.006145] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.014238] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.020581] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.028053] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.035032] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.043052] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.049222] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.057220] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.063937] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.071919] arc_prune S ffff88081e694d00 0 3296 2 0x00000080
[18298.080530] ffff88082c5cbe08 0000000000000046 ffff88082d15ce70 ffff88082c5cbfd8
[18298.089562] ffff88082c5cbfd8 ffff88082c5cbfd8 ffff88082d15ce70 ffff8800b1698a20
[18298.098581] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.107595] Call Trace:
[18298.111027] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.119036] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.125288] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.132697] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.139631] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.147627] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.153762] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.161751] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.168478] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.176463] arc_prune S ffff88081e694d00 0 3297 2 0x00000080
[18298.185077] ffff880829f87e08 0000000000000046 ffff88082d15de20 ffff880829f87fd8
[18298.194092] ffff880829f87fd8 ffff880829f87fd8 ffff88082d15de20 ffff8800b1698a80
[18298.203111] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.212144] Call Trace:
[18298.215576] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.223586] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.229833] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.237249] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.244177] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.252156] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.258280] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.266243] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.272945] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.280905] arc_prune S ffff88081e694d00 0 3298 2 0x00000080
[18298.289494] ffff88082d5f7e08 0000000000000046 ffff88082d15edd0 ffff88082d5f7fd8
[18298.298495] ffff88082d5f7fd8 ffff88082d5f7fd8 ffff88082d15edd0 ffff8800b1698ae0
[18298.307505] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.316507] Call Trace:
[18298.319923] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.327890] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.334116] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.341510] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.348420] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.356397] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.362512] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.370481] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.377176] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.385125] arc_prune S ffff88081e694d00 0 3299 2 0x00000080
[18298.393695] ffff880829a57e08 0000000000000046 ffff8808262c0000 ffff880829a57fd8
[18298.402698] ffff880829a57fd8 ffff880829a57fd8 ffff8808262c0000 ffff8800b1698b40
[18298.411694] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.420684] Call Trace:
[18298.424093] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.432076] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.438296] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.445680] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.452594] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.460558] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.466659] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.474616] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.481315] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.489280] arc_prune S ffff88081e694d00 0 3300 2 0x00000080
[18298.497846] ffff880829befe08 0000000000000046 ffff8808262c0fb0 ffff880829beffd8
[18298.506837] ffff880829beffd8 ffff880829beffd8 ffff8808262c0fb0 ffff8800b1698ba0
[18298.515824] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.524803] Call Trace:
[18298.528215] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.536195] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.542421] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.549801] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.556694] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.564655] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.570767] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.578723] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.585427] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.593383] arc_prune S ffff88081e694d00 0 3301 2 0x00000080
[18298.601968] ffff88082ca67e08 0000000000000046 ffff8808262c1f60 ffff88082ca67fd8
[18298.610975] ffff88082ca67fd8 ffff88082ca67fd8 ffff8808262c1f60 ffff8800b1698c00
[18298.619985] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.628962] Call Trace:
[18298.632376] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.640342] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.646573] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.653965] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.660856] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.668817] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.674935] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.682885] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.689581] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.697556] arc_prune S ffff88081e694d00 0 3302 2 0x00000080
[18298.706146] ffff880826e33e08 0000000000000046 ffff8808262c2f10 ffff880826e33fd8
[18298.715149] ffff880826e33fd8 ffff880826e33fd8 ffff8808262c2f10 ffff8800b1698c60
[18298.724137] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.733134] Call Trace:
[18298.736536] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.744507] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.750718] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.758112] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.765011] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.772973] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.779092] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.787046] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.793738] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.801695] arc_prune S ffff88081e694d00 0 3303 2 0x00000080
[18298.810288] ffff88082764be08 0000000000000046 ffff8808262c3ec0 ffff88082764bfd8
[18298.819292] ffff88082764bfd8 ffff88082764bfd8 ffff8808262c3ec0 ffff8800b1698cc0
[18298.828284] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.837268] Call Trace:
[18298.840682] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.848660] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.854876] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.862268] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.869167] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.877128] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.883242] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.891210] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18298.897905] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.905854] arc_prune S ffff88081e694d00 0 3304 2 0x00000080
[18298.914441] ffff88083f8f7e08 0000000000000046 ffff8808262c4e70 ffff88083f8f7fd8
[18298.923452] ffff88083f8f7fd8 ffff88083f8f7fd8 ffff8808262c4e70 ffff8800b1698d20
[18298.932465] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18298.941471] Call Trace:
[18298.944886] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.952857] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18298.959070] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18298.966463] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18298.973362] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18298.981337] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18298.987457] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18298.995421] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.002117] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.010070] arc_prune S ffff88081e694d00 0 3305 2 0x00000080
[18299.018632] ffff88082530fe08 0000000000000046 ffff8808262c5e20 ffff88082530ffd8
[18299.027630] ffff88082530ffd8 ffff88082530ffd8 ffff8808262c5e20 ffff8800b1698d80
[18299.036628] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.045623] Call Trace:
[18299.049036] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.057012] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.063232] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.070610] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.077527] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.085505] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.091617] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.099580] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.106282] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.114243] arc_prune S ffff88081e694d00 0 3306 2 0x00000080
[18299.122812] ffff880829ec7e08 0000000000000046 ffff8808262c6dd0 ffff880829ec7fd8
[18299.131816] ffff880829ec7fd8 ffff880829ec7fd8 ffff8808262c6dd0 ffff8800b1698de0
[18299.140813] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.149805] Call Trace:
[18299.153225] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.161193] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.167420] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.174796] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.181696] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.189658] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.195766] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.203717] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.210423] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.218395] arc_prune S ffff88081e694d00 0 3307 2 0x00000080
[18299.226971] ffff88082531fe08 0000000000000046 ffff88083f6f0000 ffff88082531ffd8
[18299.235973] ffff88082531ffd8 ffff88082531ffd8 ffff88083f6f0000 ffff8800b1698e40
[18299.244963] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.253959] Call Trace:
[18299.257356] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.265335] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.271561] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.278944] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.285845] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.293806] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.299915] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.307871] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.314579] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.322557] arc_prune S ffff88081e694d00 0 3308 2 0x00000080
[18299.331137] ffff880825303e08 0000000000000046 ffff88083f6f0fb0 ffff880825303fd8
[18299.340145] ffff880825303fd8 ffff880825303fd8 ffff88083f6f0fb0 ffff8800b1698ea0
[18299.349147] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.358132] Call Trace:
[18299.361557] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.369524] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.375738] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.383120] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.390022] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.397988] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.404099] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.412053] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.418747] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.426705] arc_prune S ffff88081e694d00 0 3309 2 0x00000080
[18299.435293] ffff880825687e08 0000000000000046 ffff88083f6f1f60 ffff880825687fd8
[18299.444281] ffff880825687fd8 ffff880825687fd8 ffff88083f6f1f60 ffff8800b1698f00
[18299.453273] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.462273] Call Trace:
[18299.465676] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.473644] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.479857] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.487249] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.494143] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.502104] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.508213] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.516175] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.522861] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.530818] arc_prune S ffff88081e694d00 0 3310 2 0x00000080
[18299.539391] ffff88082e17be08 0000000000000046 ffff88083f6f2f10 ffff88082e17bfd8
[18299.548402] ffff88082e17bfd8 ffff88082e17bfd8 ffff88083f6f2f10 ffff8800b1698f60
[18299.557421] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.566413] Call Trace:
[18299.569815] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.577779] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.583996] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.591385] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.598287] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.606254] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.612372] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.620338] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.627033] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.634989] arc_prune S ffff88081e694d00 0 3311 2 0x00000080
[18299.643576] ffff880827b7be08 0000000000000046 ffff88083f6f3ec0 ffff880827b7bfd8
[18299.652579] ffff880827b7bfd8 ffff880827b7bfd8 ffff88083f6f3ec0 ffff8800b1699000
[18299.661577] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.670581] Call Trace:
[18299.673989] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.681955] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.688172] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.695570] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.702471] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.710433] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.716558] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.724532] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.731226] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.739198] arc_prune S ffff88081e694d00 0 3312 2 0x00000080
[18299.747773] ffff88083feb7e08 0000000000000046 ffff88083f6f4e70 ffff88083feb7fd8
[18299.756767] ffff88083feb7fd8 ffff88083feb7fd8 ffff88083f6f4e70 ffff8800b1699060
[18299.765762] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.774750] Call Trace:
[18299.778168] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.786149] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.792365] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.799752] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.806653] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.814629] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.820741] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.828700] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.835395] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.843357] arc_prune S ffff88081e694d00 0 3313 2 0x00000080
[18299.851917] ffff88082471be08 0000000000000046 ffff88083f6f5e20 ffff88082471bfd8
[18299.860907] ffff88082471bfd8 ffff88082471bfd8 ffff88083f6f5e20 ffff8800b16990c0
[18299.869903] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.878897] Call Trace:
[18299.882319] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.890298] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18299.896523] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18299.903901] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18299.910802] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.918764] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18299.924875] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.932832] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18299.939523] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18299.947482] arc_prune S ffff88081e694d00 0 3314 2 0x00000080
[18299.956069] ffff880824713e08 0000000000000046 ffff88083f6f6dd0 ffff880824713fd8
[18299.965067] ffff880824713fd8 ffff880824713fd8 ffff88083f6f6dd0 ffff8800b1699120
[18299.974072] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18299.983058] Call Trace:
[18299.986470] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18299.994439] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.000655] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.008055] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.014948] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.022904] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.029018] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.036968] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.043663] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.051629] arc_prune S ffff88081e694d00 0 3315 2 0x00000080
[18300.060218] ffff88082af5fe08 0000000000000046 ffff880824e70000 ffff88082af5ffd8
[18300.069227] ffff88082af5ffd8 ffff88082af5ffd8 ffff880824e70000 ffff8800b1699180
[18300.078212] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.087215] Call Trace:
[18300.090628] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.098602] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.104818] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.112208] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.119116] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.127081] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.133204] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.141177] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.147872] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.155818] arc_prune S ffff88081e694d00 0 3316 2 0x00000080
[18300.164398] ffff88082562be08 0000000000000046 ffff880824e70fb0 ffff88082562bfd8
[18300.173404] ffff88082562bfd8 ffff88082562bfd8 ffff880824e70fb0 ffff8800b16991e0
[18300.182415] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.191438] Call Trace:
[18300.194851] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.202830] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.209057] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.216454] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.223364] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.231330] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.237456] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.245413] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.252115] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.260074] arc_prune S ffff88081e694d00 0 3317 2 0x00000080
[18300.268646] ffff880823fd7e08 0000000000000046 ffff880824e71f60 ffff880823fd7fd8
[18300.277645] ffff880823fd7fd8 ffff880823fd7fd8 ffff880824e71f60 ffff8800b1699240
[18300.286642] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.295648] Call Trace:
[18300.299063] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.307041] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.313277] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.320666] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.327576] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.335544] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.341653] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.349625] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.356328] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.364299] arc_prune S ffff88081e694d00 0 3318 2 0x00000080
[18300.372873] ffff880823fdbe08 0000000000000046 ffff880824e72f10 ffff880823fdbfd8
[18300.381862] ffff880823fdbfd8 ffff880823fdbfd8 ffff880824e72f10 ffff8800b16992a0
[18300.390857] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.399848] Call Trace:
[18300.403269] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.411240] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.417472] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.424858] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.431757] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.439719] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.445829] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.453788] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.460482] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.468442] arc_prune S ffff88081e694d00 0 3319 2 0x00000080
[18300.477017] ffff880823fe7e08 0000000000000046 ffff880824e73ec0 ffff880823fe7fd8
[18300.486015] ffff880823fe7fd8 ffff880823fe7fd8 ffff880824e73ec0 ffff8800b1699300
[18300.495008] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.504002] Call Trace:
[18300.507413] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.515380] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.521609] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.528994] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.535888] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.543842] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.549953] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.557911] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.564620] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.572590] arc_prune S ffff88081e694d00 0 3320 2 0x00000080
[18300.581172] ffff88082deffe08 0000000000000046 ffff880824e74e70 ffff88082defffd8
[18300.590170] ffff88082defffd8 ffff88082defffd8 ffff880824e74e70 ffff8800b1699360
[18300.599172] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.608163] Call Trace:
[18300.611580] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.619567] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.625782] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.633176] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.640086] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.648045] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.654166] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.662123] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.668810] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.676767] arc_prune S ffff88081e694d00 0 3321 2 0x00000080
[18300.685352] ffff88082523be08 0000000000000046 ffff880824e75e20 ffff88082523bfd8
[18300.694347] ffff88082523bfd8 ffff88082523bfd8 ffff880824e75e20 ffff8800b16993c0
[18300.703344] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.712346] Call Trace:
[18300.715752] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.723724] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.729936] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.737332] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.744233] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.752196] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.758305] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.766257] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.772951] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.780908] arc_prune S ffff88081e694d00 0 3322 2 0x00000080
[18300.789492] ffff88081ba43e08 0000000000000046 ffff880824e76dd0 ffff88081ba43fd8
[18300.798499] ffff88081ba43fd8 ffff88081ba43fd8 ffff880824e76dd0 ffff8800b1699420
[18300.807500] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.816513] Call Trace:
[18300.819926] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.827899] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.834126] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.841515] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.848419] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.856390] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.862509] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.870474] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.877176] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.885135] arc_prune S ffff88081e694d00 0 3323 2 0x00000080
[18300.893701] ffff8808242f7e08 0000000000000046 ffff880824e10000 ffff8808242f7fd8
[18300.902700] ffff8808242f7fd8 ffff8808242f7fd8 ffff880824e10000 ffff8800b1699480
[18300.911691] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18300.920690] Call Trace:
[18300.924106] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.932092] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18300.938310] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18300.945686] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18300.952589] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18300.960567] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18300.966678] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.974645] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18300.981350] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18300.989308] arc_prune S ffff88081e694d00 0 3324 2 0x00000080
[18300.997878] ffff8808272d3e08 0000000000000046 ffff880824e10fb0 ffff8808272d3fd8
[18301.006870] ffff8808272d3fd8 ffff8808272d3fd8 ffff880824e10fb0 ffff8800b16994e0
[18301.015866] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.024861] Call Trace:
[18301.028283] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.036266] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.042491] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.049871] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.056771] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.064732] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.070843] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.078799] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.085501] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.093470] arc_prune S ffff88081e694d00 0 3325 2 0x00000080
[18301.102049] ffff88083f6d3e08 0000000000000046 ffff880824e11f60 ffff88083f6d3fd8
[18301.111040] ffff88083f6d3fd8 ffff88083f6d3fd8 ffff880824e11f60 ffff8800b1699540
[18301.120045] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.129026] Call Trace:
[18301.132429] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.140407] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.146638] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.154032] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.160933] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.168894] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.175002] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.182964] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.189671] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.197643] arc_prune S ffff88081e694d00 0 3326 2 0x00000080
[18301.206230] ffff8808263d3e08 0000000000000046 ffff880824e12f10 ffff8808263d3fd8
[18301.215236] ffff8808263d3fd8 ffff8808263d3fd8 ffff880824e12f10 ffff8800b16995a0
[18301.224247] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.233251] Call Trace:
[18301.236671] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.244664] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.250874] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.258251] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.265151] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.273111] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.279227] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.287189] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.293880] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.301831] arc_prune S ffff88081e694d00 0 3327 2 0x00000080
[18301.310412] ffff880826bebe08 0000000000000046 ffff880824e13ec0 ffff880826bebfd8
[18301.319413] ffff880826bebfd8 ffff880826bebfd8 ffff880824e13ec0 ffff8800b1699600
[18301.328401] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.337395] Call Trace:
[18301.340800] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.348763] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.354974] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.362352] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.369259] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.377223] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.383329] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.391287] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.397980] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.405929] arc_prune S ffff88081e694d00 0 3328 2 0x00000080
[18301.414514] ffff88081c357e08 0000000000000046 ffff880824e14e70 ffff88081c357fd8
[18301.423515] ffff88081c357fd8 ffff88081c357fd8 ffff880824e14e70 ffff8800b1699660
[18301.432511] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.441517] Call Trace:
[18301.444928] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.452891] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.459103] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.466496] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.473407] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.481368] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.487476] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.495444] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.502138] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.510104] arc_prune S ffff88081e694d00 0 3329 2 0x00000080
[18301.518680] ffff88082e9fbe08 0000000000000046 ffff880824e15e20 ffff88082e9fbfd8
[18301.527686] ffff88082e9fbfd8 ffff88082e9fbfd8 ffff880824e15e20 ffff8800b16996c0
[18301.536692] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.545688] Call Trace:
[18301.549101] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.557074] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.563306] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.570691] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.577609] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.585585] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.591709] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.599679] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.606372] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.614331] arc_prune S ffff88081e694d00 0 3330 2 0x00000080
[18301.622902] ffff88083fab7e08 0000000000000046 ffff880824e16dd0 ffff88083fab7fd8
[18301.631901] ffff88083fab7fd8 ffff88083fab7fd8 ffff880824e16dd0 ffff8800b1699720
[18301.640885] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.649881] Call Trace:
[18301.653293] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.661266] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.667491] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.674876] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.681786] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.689753] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.695857] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.703805] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.710508] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.718474] arc_prune S ffff88081e694d00 0 3331 2 0x00000080
[18301.727058] ffff88082d623e08 0000000000000046 ffff880829058000 ffff88082d623fd8
[18301.736063] ffff88082d623fd8 ffff88082d623fd8 ffff880829058000 ffff8800b1699780
[18301.745063] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.754050] Call Trace:
[18301.757462] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.765436] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.771656] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.779041] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.785941] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.793901] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.800011] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.807967] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.814659] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.822620] arc_prune S ffff88081e694d00 0 3332 2 0x00000080
[18301.831200] ffff88083fb63e08 0000000000000046 ffff880829058fb0 ffff88083fb63fd8
[18301.840184] ffff88083fb63fd8 ffff88083fb63fd8 ffff880829058fb0 ffff8800b16997e0
[18301.849185] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.858170] Call Trace:
[18301.861583] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.869563] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.875777] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.883168] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.890075] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.898034] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18301.904144] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.912107] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18301.918790] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18301.926760] arc_prune S ffff88081e694d00 0 3333 2 0x00000080
[18301.935348] ffff88082a5c3e08 0000000000000046 ffff880829059f60 ffff88082a5c3fd8
[18301.944354] ffff88082a5c3fd8 ffff88082a5c3fd8 ffff880829059f60 ffff8800b1699840
[18301.953370] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18301.962367] Call Trace:
[18301.965771] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18301.973739] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18301.979953] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18301.987354] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18301.994254] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.002215] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.008324] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.016279] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.022970] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.030924] arc_prune S ffff88081e694d00 0 3334 2 0x00000080
[18302.039501] ffff88082c367e08 0000000000000046 ffff88082905af10 ffff88082c367fd8
[18302.048498] ffff88082c367fd8 ffff88082c367fd8 ffff88082905af10 ffff8800b16998a0
[18302.057504] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.066509] Call Trace:
[18302.069918] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.077891] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.084120] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.091505] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.098417] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.106380] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.112494] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.120461] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.127169] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.135130] arc_prune S ffff88081e694d00 0 3335 2 0x00000080
[18302.143711] ffff880824ecbe08 0000000000000046 ffff88082905bec0 ffff880824ecbfd8
[18302.152707] ffff880824ecbfd8 ffff880824ecbfd8 ffff88082905bec0 ffff8800b1699900
[18302.161696] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.170696] Call Trace:
[18302.174113] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.182093] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.188325] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.195711] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.202617] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.210579] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.216695] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.224660] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.231361] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.239328] arc_prune S ffff88081e694d00 0 3336 2 0x00000080
[18302.247896] ffff88082c217e08 0000000000000046 ffff88082905ce70 ffff88082c217fd8
[18302.256890] ffff88082c217fd8 ffff88082c217fd8 ffff88082905ce70 ffff8800b1699960
[18302.265886] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.274877] Call Trace:
[18302.278288] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.286262] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.292485] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.299871] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.306779] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.314743] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.320857] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.328810] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.335506] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.343471] arc_prune S ffff88081e694d00 0 3337 2 0x00000080
[18302.352031] ffff88083f7c7e08 0000000000000046 ffff88082905de20 ffff88083f7c7fd8
[18302.361010] ffff88083f7c7fd8 ffff88083f7c7fd8 ffff88082905de20 ffff8800b16999c0
[18302.369987] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.378982] Call Trace:
[18302.382404] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.390388] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.396620] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.404003] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.410907] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.418869] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.424981] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.432931] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.439634] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.447595] arc_prune S ffff88081e694d00 0 3338 2 0x00000080
[18302.456174] ffff88083f7a7e08 0000000000000046 ffff88082905edd0 ffff88083f7a7fd8
[18302.465186] ffff88083f7a7fd8 ffff88083f7a7fd8 ffff88082905edd0 ffff8800b1699a20
[18302.474188] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.483181] Call Trace:
[18302.486581] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.494554] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.500763] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.508148] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.515041] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.522999] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.529125] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.537090] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.543784] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.551745] arc_prune S ffff88081e694d00 0 3339 2 0x00000080
[18302.560330] ffff880824fbbe08 0000000000000046 ffff88082d2c8000 ffff880824fbbfd8
[18302.569324] ffff880824fbbfd8 ffff880824fbbfd8 ffff88082d2c8000 ffff8800b1699a80
[18302.578333] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.587345] Call Trace:
[18302.590747] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.598726] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.604943] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.612337] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.619249] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.627214] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.633335] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.641299] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.647994] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.655940] arc_prune S ffff88081e694d00 0 3340 2 0x00000080
[18302.664535] ffff880826f47e08 0000000000000046 ffff88082d2c8fb0 ffff880826f47fd8
[18302.673542] ffff880826f47fd8 ffff880826f47fd8 ffff88082d2c8fb0 ffff8800b1699ae0
[18302.682552] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.691564] Call Trace:
[18302.694977] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.702947] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.709165] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.716561] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.723471] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.731443] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.737557] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.745525] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.752220] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.760178] arc_prune S ffff88081e694d00 0 3341 2 0x00000080
[18302.768750] ffff880824e4be08 0000000000000046 ffff88082d2c9f60 ffff880824e4bfd8
[18302.777744] ffff880824e4bfd8 ffff880824e4bfd8 ffff88082d2c9f60 ffff8800b1699b40
[18302.786745] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.795743] Call Trace:
[18302.799156] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.807140] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.813358] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.820752] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.827662] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.835638] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.841747] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.849713] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.856402] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.864361] arc_prune S ffff88081e694d00 0 3342 2 0x00000080
[18302.872941] ffff880824e47e08 0000000000000046 ffff88082d2caf10 ffff880824e47fd8
[18302.881944] ffff880824e47fd8 ffff880824e47fd8 ffff88082d2caf10 ffff8800b1699ba0
[18302.890944] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18302.899944] Call Trace:
[18302.903350] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.911330] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18302.917554] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18302.924938] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18302.931836] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18302.939796] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18302.945909] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.953871] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18302.960564] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18302.968540] arc_prune S ffff88081e694d00 0 3343 2 0x00000080
[18302.977125] ffff880826b77e08 0000000000000046 ffff88082d2cbec0 ffff880826b77fd8
[18302.986130] ffff880826b77fd8 ffff880826b77fd8 ffff88082d2cbec0 ffff8800b1699c00
[18302.995140] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.004135] Call Trace:
[18303.007543] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.015508] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.021732] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.029118] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.036012] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.043971] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.050082] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.058039] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.064737] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.072703] arc_prune S ffff88081e694d00 0 3344 2 0x00000080
[18303.081288] ffff88082d163e08 0000000000000046 ffff88082d2cce70 ffff88082d163fd8
[18303.090289] ffff88082d163fd8 ffff88082d163fd8 ffff88082d2cce70 ffff8800b1699c60
[18303.099285] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.108277] Call Trace:
[18303.111691] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.119667] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.125881] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.133267] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.140168] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.148131] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.154251] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.162221] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.168908] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.176865] arc_prune S ffff88081e694d00 0 3345 2 0x00000080
[18303.185471] ffff88082e677e08 0000000000000046 ffff88082d2cde20 ffff88082e677fd8
[18303.194488] ffff88082e677fd8 ffff88082e677fd8 ffff88082d2cde20 ffff8800b1699cc0
[18303.203489] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.212505] Call Trace:
[18303.215908] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.223876] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.230097] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.237490] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.244400] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.252369] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.258484] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.266448] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.273134] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.281100] arc_prune S ffff88081e694d00 0 3346 2 0x00000080
[18303.289687] ffff88082e63fe08 0000000000000046 ffff88082d2cedd0 ffff88082e63ffd8
[18303.298693] ffff88082e63ffd8 ffff88082e63ffd8 ffff88082d2cedd0 ffff8800b1699d20
[18303.307698] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.316699] Call Trace:
[18303.320107] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.328076] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.334293] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.341699] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.348601] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.356562] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.362685] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.370656] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.377358] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.385314] arc_prune S ffff88081e694d00 0 3347 2 0x00000080
[18303.393883] ffff880827f97e08 0000000000000046 ffff880829450000 ffff880827f97fd8
[18303.402876] ffff880827f97fd8 ffff880827f97fd8 ffff880829450000 ffff8800b1699d80
[18303.411873] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.420865] Call Trace:
[18303.424271] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.432239] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.438458] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.445841] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.452746] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.460722] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.466832] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.474788] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.481482] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.489447] arc_prune S ffff88081e694d00 0 3348 2 0x00000080
[18303.498003] ffff8808272a3e08 0000000000000046 ffff880829450fb0 ffff8808272a3fd8
[18303.506981] ffff8808272a3fd8 ffff8808272a3fd8 ffff880829450fb0 ffff8800b1699de0
[18303.515954] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.524956] Call Trace:
[18303.528370] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.536350] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.542574] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.549956] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.556856] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.564815] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.570925] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.578883] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.585584] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.593548] arc_prune S ffff88081e694d00 0 3349 2 0x00000080
[18303.602140] ffff88082d14fe08 0000000000000046 ffff880829451f60 ffff88082d14ffd8
[18303.611130] ffff88082d14ffd8 ffff88082d14ffd8 ffff880829451f60 ffff8800b1699e40
[18303.620142] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.629135] Call Trace:
[18303.632544] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.640514] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.646738] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.654123] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.661024] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.668978] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.675087] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.683042] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.689741] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.697703] arc_prune S ffff88081e694d00 0 3350 2 0x00000080
[18303.706289] ffff88082d157e08 0000000000000046 ffff880829452f10 ffff88082d157fd8
[18303.715294] ffff88082d157fd8 ffff88082d157fd8 ffff880829452f10 ffff8800b1699ea0
[18303.724298] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.733302] Call Trace:
[18303.736721] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.744703] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.750906] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.758295] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.765196] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.773167] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.779276] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.787236] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.793927] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.801880] arc_prune S ffff88081e694d00 0 3351 2 0x00000080
[18303.810469] ffff88082e633e08 0000000000000046 ffff880829453ec0 ffff88082e633fd8
[18303.819472] ffff88082e633fd8 ffff88082e633fd8 ffff880829453ec0 ffff8800b1699f00
[18303.828484] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.837491] Call Trace:
[18303.840904] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.848874] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.855083] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.862476] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.869377] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.877340] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.883450] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.891408] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18303.898101] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.906057] arc_prune S ffff88081e694d00 0 3352 2 0x00000080
[18303.914637] ffff88082d4cfe08 0000000000000046 ffff880829454e70 ffff88082d4cffd8
[18303.923646] ffff88082d4cffd8 ffff88082d4cffd8 ffff880829454e70 ffff8800b1699f60
[18303.932642] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18303.941638] Call Trace:
[18303.945038] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.953008] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18303.959224] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18303.966621] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18303.973520] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18303.981474] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18303.987581] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18303.995545] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.002238] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.010196] arc_prune S ffff88081e694d00 0 3353 2 0x00000080
[18304.018760] ffff88081e77be08 0000000000000046 ffff880829455e20 ffff88081e77bfd8
[18304.027765] ffff88081e77bfd8 ffff88081e77bfd8 ffff880829455e20 ffff8800b169a000
[18304.036765] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.045764] Call Trace:
[18304.049180] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.057150] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.063372] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.070761] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.077656] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.085615] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.091723] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.099684] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.106384] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.114350] arc_prune S ffff88081e694d00 0 3354 2 0x00000080
[18304.122918] ffff88081baefe08 0000000000000046 ffff880829456dd0 ffff88081baeffd8
[18304.131915] ffff88081baeffd8 ffff88081baeffd8 ffff880829456dd0 ffff8800b169a060
[18304.140898] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.149888] Call Trace:
[18304.153302] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.161278] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.167500] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.174882] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.181770] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.189742] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.195848] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.203803] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.210501] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.218454] arc_prune S ffff88081e694d00 0 3355 2 0x00000080
[18304.227010] ffff88082abcbe08 0000000000000046 ffff880825ab8000 ffff88082abcbfd8
[18304.235998] ffff88082abcbfd8 ffff88082abcbfd8 ffff880825ab8000 ffff8800b169a0c0
[18304.244995] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.253983] Call Trace:
[18304.257402] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.265375] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.271606] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.278984] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.285883] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.293843] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.299947] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.307903] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.314598] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.322562] arc_prune S ffff88081e694d00 0 3356 2 0x00000080
[18304.331144] ffff880827e5fe08 0000000000000046 ffff880825ab8fb0 ffff880827e5ffd8
[18304.340138] ffff880827e5ffd8 ffff880827e5ffd8 ffff880825ab8fb0 ffff8800b169a120
[18304.349126] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.358115] Call Trace:
[18304.361528] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.369502] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.375717] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.383100] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.390001] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.397961] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.404072] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.412029] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.418731] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.426690] arc_prune S ffff88081e694d00 0 3357 2 0x00000080
[18304.435281] ffff880829b0be08 0000000000000046 ffff880825ab9f60 ffff880829b0bfd8
[18304.444281] ffff880829b0bfd8 ffff880829b0bfd8 ffff880825ab9f60 ffff8800b169a180
[18304.453276] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.462279] Call Trace:
[18304.465706] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.473693] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.479913] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.487307] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.494207] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.502180] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.508297] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.516253] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.522938] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.530894] arc_prune S ffff88081e694d00 0 3358 2 0x00000080
[18304.539483] ffff880829a47e08 0000000000000046 ffff880825abaf10 ffff880829a47fd8
[18304.548478] ffff880829a47fd8 ffff880829a47fd8 ffff880825abaf10 ffff8800b169a1e0
[18304.557492] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.566502] Call Trace:
[18304.569906] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.577879] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.584095] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.591485] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.598385] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.606344] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.612460] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.620426] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.627111] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.635061] arc_prune S ffff88081e694d00 0 3359 2 0x00000080
[18304.643633] ffff880829a4be08 0000000000000046 ffff880825abbec0 ffff880829a4bfd8
[18304.652636] ffff880829a4bfd8 ffff880829a4bfd8 ffff880825abbec0 ffff8800b169a240
[18304.661635] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.670642] Call Trace:
[18304.674058] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.682029] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.688254] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.695648] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.702550] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.710511] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.716634] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.724605] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.731300] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.739257] arc_prune S ffff88081e694d00 0 3360 2 0x00000080
[18304.747827] ffff8808266cbe08 0000000000000046 ffff880825abce70 ffff8808266cbfd8
[18304.756822] ffff8808266cbfd8 ffff8808266cbfd8 ffff880825abce70 ffff8800b169a2a0
[18304.765819] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.774809] Call Trace:
[18304.778220] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.786198] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.792420] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.799819] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.806734] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.814701] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.820812] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.828769] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.835463] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.843423] arc_prune S ffff88081e694d00 0 3361 2 0x00000080
[18304.851999] ffff880824e5be08 0000000000000046 ffff880825abde20 ffff880824e5bfd8
[18304.860998] ffff880824e5bfd8 ffff880824e5bfd8 ffff880825abde20 ffff8800b169a300
[18304.870002] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.878990] Call Trace:
[18304.882404] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.890378] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18304.896604] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18304.903990] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18304.910894] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18304.918864] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18304.924975] [<ffffffff810bfde3>] ? finish_task_switch+0x53/0x180
[18304.932452] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.940416] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18304.947110] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18304.955061] arc_prune S ffff88081e694d00 0 3362 2 0x00000080
[18304.963640] ffff88083f8ffe08 0000000000000046 ffff880825abedd0 ffff88083f8fffd8
[18304.972647] ffff88083f8fffd8 ffff88083f8fffd8 ffff880825abedd0 ffff8800b169a360
[18304.981654] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18304.990649] Call Trace:
[18304.994060] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.002021] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.008251] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.015635] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.022538] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.030504] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.036619] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.044578] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.051275] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.059233] arc_prune S ffff88081e694d00 0 3363 2 0x00000080
[18305.067800] ffff88083f89fe08 0000000000000046 ffff880829680000 ffff88083f89ffd8
[18305.076807] ffff88083f89ffd8 ffff88083f89ffd8 ffff880829680000 ffff8800b169a3c0
[18305.085802] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.094803] Call Trace:
[18305.098199] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.106178] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.112398] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.119788] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.126696] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.134659] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.140782] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.148747] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.155440] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.163401] arc_prune S ffff88081e694d00 0 3364 2 0x00000080
[18305.171978] ffff88083f8e7e08 0000000000000046 ffff880829680fb0 ffff88083f8e7fd8
[18305.180974] ffff88083f8e7fd8 ffff88083f8e7fd8 ffff880829680fb0 ffff8800b169a420
[18305.189980] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.198976] Call Trace:
[18305.202372] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.210332] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.216551] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.223925] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.230814] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.238775] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.244882] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.252836] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.259536] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.267502] arc_prune S ffff88081e694d00 0 3365 2 0x00000080
[18305.276067] ffff88082d683e08 0000000000000046 ffff880829681f60 ffff88082d683fd8
[18305.285053] ffff88082d683fd8 ffff88082d683fd8 ffff880829681f60 ffff8800b169a480
[18305.294046] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.303036] Call Trace:
[18305.306453] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.314430] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.320659] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.328035] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.334932] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.342894] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.349003] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.356958] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.363648] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.371602] arc_prune S ffff88081e694d00 0 3366 2 0x00000080
[18305.380181] ffff880829433e08 0000000000000046 ffff880829682f10 ffff880829433fd8
[18305.389193] ffff880829433fd8 ffff880829433fd8 ffff880829682f10 ffff8800b169a4e0
[18305.398200] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.407201] Call Trace:
[18305.410609] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.418580] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.424809] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.432191] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.439091] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.447052] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.453163] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.461110] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.467805] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.475765] arc_prune S ffff88081e694d00 0 3367 2 0x00000080
[18305.484346] ffff88082942fe08 0000000000000046 ffff880829683ec0 ffff88082942ffd8
[18305.493343] ffff88082942ffd8 ffff88082942ffd8 ffff880829683ec0 ffff8800b169a540
[18305.502329] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.511323] Call Trace:
[18305.514732] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.522710] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.528927] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.536320] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.543217] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.551192] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.557311] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.565265] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.571959] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.579911] arc_prune S ffff88081e694d00 0 3368 2 0x00000080
[18305.588497] ffff88083f8d7e08 0000000000000046 ffff880829684e70 ffff88083f8d7fd8
[18305.597520] ffff88083f8d7fd8 ffff88083f8d7fd8 ffff880829684e70 ffff8800b169a5a0
[18305.606522] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.615521] Call Trace:
[18305.618934] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.626905] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.633119] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.640505] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.647409] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.655371] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.661491] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.669457] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.676151] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.684108] arc_prune S ffff88081e694d00 0 3369 2 0x00000080
[18305.692684] ffff88083f8dbe08 0000000000000046 ffff880829685e20 ffff88083f8dbfd8
[18305.701697] ffff88083f8dbfd8 ffff88083f8dbfd8 ffff880829685e20 ffff8800b169a600
[18305.710702] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.719690] Call Trace:
[18305.723099] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.731065] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.737289] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.744679] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.751579] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.759549] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.765659] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.773621] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.780329] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.788294] arc_prune S ffff88081e694d00 0 3370 2 0x00000080
[18305.796867] ffff88082421be08 0000000000000046 ffff880829686dd0 ffff88082421bfd8
[18305.805860] ffff88082421bfd8 ffff88082421bfd8 ffff880829686dd0 ffff8800b169a660
[18305.814858] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.823844] Call Trace:
[18305.827249] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.835226] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.841450] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.848833] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.855736] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.863701] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.869816] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.877787] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.884485] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.892451] arc_prune S ffff88081e694d00 0 3371 2 0x00000080
[18305.901017] ffff880824217e08 0000000000000046 ffff88081bb48000 ffff880824217fd8
[18305.910023] ffff880824217fd8 ffff880824217fd8 ffff88081bb48000 ffff8800b169a6c0
[18305.919021] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18305.928028] Call Trace:
[18305.931446] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.939417] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18305.945640] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18305.953022] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18305.959924] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18305.967884] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18305.973993] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.981943] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18305.988644] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18305.996603] arc_prune S ffff88081e694d00 0 3372 2 0x00000080
[18306.005179] ffff88082abd7e08 0000000000000046 ffff88081bb48fb0 ffff88082abd7fd8
[18306.014187] ffff88082abd7fd8 ffff88082abd7fd8 ffff88081bb48fb0 ffff8800b169a720
[18306.023194] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.032185] Call Trace:
[18306.035600] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.043570] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.049795] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.057180] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.064081] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.072043] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.078153] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.086108] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.092806] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.100779] arc_prune S ffff88081e694d00 0 3373 2 0x00000080
[18306.109351] ffff88081e7b7e08 0000000000000046 ffff88081bb49f60 ffff88081e7b7fd8
[18306.118359] ffff88081e7b7fd8 ffff88081e7b7fd8 ffff88081bb49f60 ffff8800b169a780
[18306.127352] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.136338] Call Trace:
[18306.139756] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.147743] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.153963] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.161355] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.168266] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.176227] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.182336] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.190294] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.196993] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.204950] arc_prune S ffff88081e694d00 0 3374 2 0x00000080
[18306.213544] ffff88082c32fe08 0000000000000046 ffff88081bb4af10 ffff88082c32ffd8
[18306.222552] ffff88082c32ffd8 ffff88082c32ffd8 ffff88081bb4af10 ffff8800b169a7e0
[18306.231563] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.240571] Call Trace:
[18306.243975] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.251944] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.258160] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.265560] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.272449] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.280418] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.286537] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.294502] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.301198] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.309153] arc_prune S ffff88081e694d00 0 3375 2 0x00000080
[18306.317732] ffff88082d063e08 0000000000000046 ffff88081bb4bec0 ffff88082d063fd8
[18306.326751] ffff88082d063fd8 ffff88082d063fd8 ffff88081bb4bec0 ffff8800b169a840
[18306.335765] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.344775] Call Trace:
[18306.348182] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.356146] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.362372] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.369772] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.376663] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.384632] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.390756] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.398723] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.405417] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.413374] arc_prune S ffff88081e694d00 0 3376 2 0x00000080
[18306.421948] ffff88082af23e08 0000000000000046 ffff88081bb4ce70 ffff88082af23fd8
[18306.430945] ffff88082af23fd8 ffff88082af23fd8 ffff88081bb4ce70 ffff8800b169a8a0
[18306.439928] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.448925] Call Trace:
[18306.452343] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.460312] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.466542] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.473926] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.480841] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.488794] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.494912] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.502867] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.509561] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.517522] arc_prune S ffff88081e694d00 0 3377 2 0x00000080
[18306.526095] ffff880825763e08 0000000000000046 ffff88081bb4de20 ffff880825763fd8
[18306.535073] ffff880825763fd8 ffff880825763fd8 ffff88081bb4de20 ffff8800b169a900
[18306.544067] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.553066] Call Trace:
[18306.556480] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.564450] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.570675] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.578060] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.584959] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.592921] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.599030] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.606984] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.613676] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.621637] arc_prune S ffff88081e694d00 0 3378 2 0x00000080
[18306.630227] ffff88082628fe08 0000000000000046 ffff88081bb4edd0 ffff88082628ffd8
[18306.639226] ffff88082628ffd8 ffff88082628ffd8 ffff88081bb4edd0 ffff8800b169a960
[18306.648215] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.657198] Call Trace:
[18306.660605] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.668571] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.674801] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.682178] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.689073] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.697032] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.703142] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.711098] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.717807] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.725787] arc_prune S ffff88081e694d00 0 3379 2 0x00000080
[18306.734371] ffff88082944fe08 0000000000000046 ffff880826798000 ffff88082944ffd8
[18306.743359] ffff88082944ffd8 ffff88082944ffd8 ffff880826798000 ffff8800b169a9c0
[18306.752356] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.761356] Call Trace:
[18306.764781] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.772767] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.778990] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.786383] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.793292] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.801264] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.807373] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.815332] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.822027] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.829988] arc_prune S ffff88081e694d00 0 3380 2 0x00000080
[18306.838577] ffff8808256ebe08 0000000000000046 ffff880826798fb0 ffff8808256ebfd8
[18306.847591] ffff8808256ebfd8 ffff8808256ebfd8 ffff880826798fb0 ffff8800b169aa20
[18306.856597] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.865599] Call Trace:
[18306.868997] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.876965] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.883174] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.890560] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18306.897463] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.905433] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18306.911544] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.919506] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18306.926192] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18306.934134] arc_prune S ffff88081e694d00 0 3381 2 0x00000080
[18306.942703] ffff8808252a3e08 0000000000000046 ffff880826799f60 ffff8808252a3fd8
[18306.951695] ffff8808252a3fd8 ffff8808252a3fd8 ffff880826799f60 ffff8800b169aa80
[18306.960701] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18306.969700] Call Trace:
[18306.973106] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18306.981077] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18306.987297] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18306.994682] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.001590] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.009561] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.015674] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.023634] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.030327] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.038286] arc_prune S ffff88081e694d00 0 3382 2 0x00000080
[18307.046874] ffff88082945be08 0000000000000046 ffff88082679af10 ffff88082945bfd8
[18307.055877] ffff88082945bfd8 ffff88082945bfd8 ffff88082679af10 ffff8800b169aae0
[18307.064870] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.073873] Call Trace:
[18307.077292] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.085281] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.091498] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.098889] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.105792] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.113765] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.119887] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.127847] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.134554] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.142512] arc_prune S ffff88081e694d00 0 3383 2 0x00000080
[18307.151072] ffff8808276bfe08 0000000000000046 ffff88082679bec0 ffff8808276bffd8
[18307.160054] ffff8808276bffd8 ffff8808276bffd8 ffff88082679bec0 ffff8800b169ab40
[18307.169045] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.178041] Call Trace:
[18307.181449] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.189423] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.195639] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.203023] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.209921] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.217888] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.224001] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.231961] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.238665] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.246632] arc_prune S ffff88081e694d00 0 3384 2 0x00000080
[18307.255199] ffff880823e23e08 0000000000000046 ffff88082679ce70 ffff880823e23fd8
[18307.264191] ffff880823e23fd8 ffff880823e23fd8 ffff88082679ce70 ffff8800b169aba0
[18307.273187] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.282160] Call Trace:
[18307.285568] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.293539] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.299759] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.307144] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.314043] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.322001] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.328104] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.336058] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.342753] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.350710] arc_prune S ffff88081e694d00 0 3385 2 0x00000080
[18307.359296] ffff880823e1fe08 0000000000000046 ffff88082679de20 ffff880823e1ffd8
[18307.368307] ffff880823e1ffd8 ffff880823e1ffd8 ffff88082679de20 ffff8800b169ac00
[18307.377302] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.386299] Call Trace:
[18307.389717] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.397696] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.403912] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.411302] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.418202] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.426161] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.432281] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.440253] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.446939] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.454889] arc_prune S ffff88081e694d00 0 3386 2 0x00000080
[18307.463473] ffff88081bb9fe08 0000000000000046 ffff88082679edd0 ffff88081bb9ffd8
[18307.472456] ffff88081bb9ffd8 ffff88081bb9ffd8 ffff88082679edd0 ffff8800b169ac60
[18307.481459] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.490455] Call Trace:
[18307.493869] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.501851] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.508064] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.515449] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.522365] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.530336] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.536445] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.544400] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.551092] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.559046] arc_prune S ffff88081e694d00 0 3387 2 0x00000080
[18307.567641] ffff88082433be08 0000000000000046 ffff880825248000 ffff88082433bfd8
[18307.576654] ffff88082433bfd8 ffff88082433bfd8 ffff880825248000 ffff8800b169acc0
[18307.585660] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.594670] Call Trace:
[18307.598079] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.606048] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.612269] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.619653] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.626563] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.634531] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.640649] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.648613] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.655292] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.663257] arc_prune S ffff88081e694d00 0 3388 2 0x00000080
[18307.671833] ffff88083f8b3e08 0000000000000046 ffff880825248fb0 ffff88083f8b3fd8
[18307.680846] ffff88083f8b3fd8 ffff88083f8b3fd8 ffff880825248fb0 ffff8800b169ad20
[18307.689855] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.698851] Call Trace:
[18307.702265] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.710235] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.716451] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.723848] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.730756] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.738729] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.744841] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.752812] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.759506] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.767468] arc_prune S ffff88081e694d00 0 3389 2 0x00000080
[18307.776035] ffff880829023e08 0000000000000046 ffff880825249f60 ffff880829023fd8
[18307.785034] ffff880829023fd8 ffff880829023fd8 ffff880825249f60 ffff8800b169ad80
[18307.794023] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.803025] Call Trace:
[18307.806433] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.814404] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.820621] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.828006] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.834908] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.842877] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.848985] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.856941] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.863622] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.871588] arc_prune S ffff88081e694d00 0 3390 2 0x00000080
[18307.880158] ffff88083f8cfe08 0000000000000046 ffff88082524af10 ffff88083f8cffd8
[18307.889147] ffff88083f8cffd8 ffff88083f8cffd8 ffff88082524af10 ffff8800b169ade0
[18307.898133] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18307.907120] Call Trace:
[18307.910533] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.918511] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18307.924749] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18307.932135] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18307.939036] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18307.946996] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18307.953106] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.961057] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18307.967758] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18307.975716] arc_prune S ffff88081e694d00 0 3391 2 0x00000080
[18307.984295] ffff8808283bbe08 0000000000000046 ffff88082524bec0 ffff8808283bbfd8
[18307.993296] ffff8808283bbfd8 ffff8808283bbfd8 ffff88082524bec0 ffff8800b169ae40
[18308.002295] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18308.011282] Call Trace:
[18308.014688] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.022668] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18308.028886] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18308.036270] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18308.043169] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.051131] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18308.057239] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.065187] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18308.071901] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.079874] arc_prune S ffff88081e694d00 0 3392 2 0x00000080
[18308.088463] ffff880825263e08 0000000000000046 ffff88082524ce70 ffff880825263fd8
[18308.097465] ffff880825263fd8 ffff880825263fd8 ffff88082524ce70 ffff8800b169aea0
[18308.106466] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18308.115473] Call Trace:
[18308.118889] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.126869] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18308.133083] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18308.140483] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18308.147384] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.155352] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18308.161467] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.169421] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18308.176114] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.184072] arc_prune S ffff88081e694d00 0 3393 2 0x00000080
[18308.192658] ffff88083f8afe08 0000000000000046 ffff88082524de20 ffff88083f8affd8
[18308.201658] ffff88083f8affd8 ffff88083f8affd8 ffff88082524de20 ffff8800b169af00
[18308.210667] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18308.219676] Call Trace:
[18308.223079] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.231049] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18308.237271] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18308.244664] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18308.251564] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.259522] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18308.265633] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.273594] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18308.280292] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.288242] arc_prune S ffff88081e694d00 0 3394 2 0x00000080
[18308.296825] ffff880829e67e08 0000000000000046 ffff88082524edd0 ffff880829e67fd8
[18308.305834] ffff880829e67fd8 ffff880829e67fd8 ffff88082524edd0 ffff8800b169af60
[18308.314839] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18308.323827] Call Trace:
[18308.327232] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.335202] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18308.341417] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18308.348809] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18308.355710] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.363671] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18308.369792] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.377757] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18308.384448] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.392405] arc_prune S ffff88081e694d00 0 3395 2 0x00000080
[18308.400996] ffff88082aa6fe08 0000000000000046 ffff88083f840000 ffff88082aa6ffd8
[18308.409990] ffff88082aa6ffd8 ffff88082aa6ffd8 ffff88083f840000 ffff8800b169b000
[18308.418986] ffff88081e694d90 ffffffffa08254b0 ffff88081e694db0 ffff88081e694d00
[18308.427983] Call Trace:
[18308.431399] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.439378] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18308.445603] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18308.452981] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18308.459898] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18308.467873] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18308.473977] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.481928] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18308.488621] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.496576] arc_reclaim S 000000000000f888 0 3396 2 0x00000080
[18308.505143] ffff88082d61fd30 0000000000000046 ffff88083f840fb0 ffff88082d61ffd8
[18308.514130] ffff88082d61ffd8 ffff88082d61ffd8 ffff88083f840fb0 ffff8804b96ec000
[18308.523122] ffff88082d61fd60 000000010112c7bf ffff8804b96ec000 000000000000f888
[18308.532120] Call Trace:
[18308.535527] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18308.541757] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18308.548939] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18308.556310] [<ffffffff810b1816>] ? prepare_to_wait_exclusive+0x56/0x90
[18308.564377] [<ffffffffa08297ba>] __cv_timedwait_common+0xba/0x160 [spl]
[18308.572531] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18308.579713] [<ffffffffa0829893>] __cv_timedwait_sig+0x13/0x20 [spl]
[18308.587506] [<ffffffffa08aea19>] arc_reclaim_thread+0x109/0x210 [zfs]
[18308.595473] [<ffffffffa08ae910>] ? arc_shrink+0xc0/0xc0 [zfs]
[18308.602668] [<ffffffffa0824851>] thread_generic_wrapper+0x71/0x80 [spl]
[18308.610817] [<ffffffffa08247e0>] ? __thread_exit+0x20/0x20 [spl]
[18308.618280] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18308.624387] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.632344] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18308.639029] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.646974] arc_user_evicts S 000000000000f888 0 3397 2 0x00000080
[18308.655540] ffff8808242dfd40 0000000000000046 ffff88083f841f60 ffff8808242dffd8
[18308.664530] ffff8808242dffd8 ffff8808242dffd8 ffff88083f841f60 ffff8804b9068000
[18308.673506] ffff8808242dfd70 000000010112c7bf ffff8804b9068000 000000000000f888
[18308.682484] Call Trace:
[18308.685874] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18308.692060] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18308.699221] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18308.706584] [<ffffffff810b1816>] ? prepare_to_wait_exclusive+0x56/0x90
[18308.714634] [<ffffffffa08297ba>] __cv_timedwait_common+0xba/0x160 [spl]
[18308.722788] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18308.729956] [<ffffffffa08a9240>] ? hdr_recl+0x50/0x50 [zfs]
[18308.736927] [<ffffffffa0829893>] __cv_timedwait_sig+0x13/0x20 [spl]
[18308.744677] [<ffffffffa08a92ed>] arc_user_evicts_thread+0xad/0x110 [zfs]
[18308.752907] [<ffffffffa0824851>] thread_generic_wrapper+0x71/0x80 [spl]
[18308.761031] [<ffffffffa08247e0>] ? __thread_exit+0x20/0x20 [spl]
[18308.768467] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18308.774551] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.782485] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18308.789137] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.797052] l2arc_feed S 000000000000f888 0 3398 2 0x00000080
[18308.805602] ffff88083f87bc70 0000000000000046 ffff88083f842f10 ffff88083f87bfd8
[18308.814572] ffff88083f87bfd8 ffff88083f87bfd8 ffff88083f842f10 ffff880179130000
[18308.823533] ffff88083f87bca0 000000010112c75c ffff880179130000 000000000000f888
[18308.832487] Call Trace:
[18308.835862] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18308.842049] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18308.849205] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18308.856555] [<ffffffff810b1816>] ? prepare_to_wait_exclusive+0x56/0x90
[18308.864582] [<ffffffffa08297ba>] __cv_timedwait_common+0xba/0x160 [spl]
[18308.872689] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18308.879856] [<ffffffffa08abc50>] ? l2arc_evict+0x280/0x280 [zfs]
[18308.887299] [<ffffffffa0829893>] __cv_timedwait_sig+0x13/0x20 [spl]
[18308.895033] [<ffffffffa08abcc8>] l2arc_feed_thread+0x78/0xbe0 [zfs]
[18308.902769] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18308.909829] [<ffffffffa08abc50>] ? l2arc_evict+0x280/0x280 [zfs]
[18308.917271] [<ffffffffa082275a>] ? spl_kmem_free+0x2a/0x40 [spl]
[18308.924716] [<ffffffff811dd173>] ? kfree+0x103/0x140
[18308.930987] [<ffffffffa08abc50>] ? l2arc_evict+0x280/0x280 [zfs]
[18308.938424] [<ffffffffa0824851>] thread_generic_wrapper+0x71/0x80 [spl]
[18308.946547] [<ffffffffa08247e0>] ? __thread_exit+0x20/0x20 [spl]
[18308.953977] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18308.960046] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.967959] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18308.974619] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18308.982544] z_unmount S ffff88081e694f00 0 3399 2 0x00000080
[18308.991058] ffff8808242e7e08 0000000000000046 ffff88083f843ec0 ffff8808242e7fd8
[18309.000007] ffff8808242e7fd8 ffff8808242e7fd8 ffff88083f843ec0 ffff8800b169b060
[18309.008960] ffff88081e694f90 ffffffffa08254b0 ffff88081e694fb0 ffff88081e694f00
[18309.017904] Call Trace:
[18309.021265] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18309.029194] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.035375] [<ffffffffa08258aa>] taskq_thread+0x3fa/0x440 [spl]
[18309.042716] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18309.049583] [<ffffffffa08254b0>] ? taskq_thread_spawn+0x60/0x60 [spl]
[18309.057510] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.063593] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.071522] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.078174] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.086090] cfs_rh_00 S ffff88042bc2ce30 0 3411 2 0x00000080
[18309.094645] ffff880826763e28 0000000000000046 ffff88083f844e70 ffff880826763fd8
[18309.103615] ffff880826763fd8 ffff880826763fd8 ffff88083f844e70 ffff88042bc2ce00
[18309.112577] ffff88042bc2ce18 ffff88042bc2ce10 0000000000000000 ffff88042bc2ce30
[18309.121550] Call Trace:
[18309.124920] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.131121] [<ffffffffa0ba5de8>] cfs_wi_scheduler+0x2f8/0x440 [libcfs]
[18309.139151] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18309.146320] [<ffffffffa0ba5af0>] ? cfs_wi_schedule+0x190/0x190 [libcfs]
[18309.154449] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.160535] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.168471] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.175142] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.183063] cfs_rh_01 S ffff88042bc2ce30 0 3412 2 0x00000080
[18309.191604] ffff8808273f3e28 0000000000000046 ffff88083f845e20 ffff8808273f3fd8
[18309.200571] ffff8808273f3fd8 ffff8808273f3fd8 ffff88083f845e20 ffff88042bc2ce00
[18309.209549] ffff88042bc2ce18 ffff88042bc2ce10 0000000000000000 ffff88042bc2ce30
[18309.218521] Call Trace:
[18309.221904] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.228104] [<ffffffffa0ba5de8>] cfs_wi_scheduler+0x2f8/0x440 [libcfs]
[18309.236149] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18309.243329] [<ffffffffa0ba5af0>] ? cfs_wi_schedule+0x190/0x190 [libcfs]
[18309.251483] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.257587] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.265529] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.272219] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.280173] cfs_rh_02 S ffff88042bc2ce30 0 3413 2 0x00000080
[18309.288757] ffff88082a757e28 0000000000000046 ffff88083f846dd0 ffff88082a757fd8
[18309.297753] ffff88082a757fd8 ffff88082a757fd8 ffff88083f846dd0 ffff88042bc2ce00
[18309.306745] ffff88042bc2ce18 ffff88042bc2ce10 0000000000000000 ffff88042bc2ce30
[18309.315757] Call Trace:
[18309.319169] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.325403] [<ffffffffa0ba5de8>] cfs_wi_scheduler+0x2f8/0x440 [libcfs]
[18309.333480] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18309.340679] [<ffffffffa0ba5af0>] ? cfs_wi_schedule+0x190/0x190 [libcfs]
[18309.348840] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.354971] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.362943] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.369638] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.377596] cfs_rh_03 S ffff88042bc2ce30 0 3414 2 0x00000080
[18309.386163] ffff88081badbe28 0000000000000046 ffff88082d220000 ffff88081badbfd8
[18309.395156] ffff88081badbfd8 ffff88081badbfd8 ffff88082d220000 ffff88042bc2ce00
[18309.404144] ffff88042bc2ce18 ffff88042bc2ce10 0000000000000000 ffff88042bc2ce30
[18309.413126] Call Trace:
[18309.416533] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.422766] [<ffffffffa0ba5de8>] cfs_wi_scheduler+0x2f8/0x440 [libcfs]
[18309.430838] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18309.438037] [<ffffffffa0ba5af0>] ? cfs_wi_schedule+0x190/0x190 [libcfs]
[18309.446194] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.452305] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.460265] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.466977] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.474939] obd_zombid S 0000000000000000 0 3419 2 0x00000080
[18309.483534] ffff8808247ebe40 0000000000000046 ffff88082d222f10 ffff8808247ebfd8
[18309.492525] ffff8808247ebfd8 ffff8808247ebfd8 ffff88082d222f10 ffff88082d222f10
[18309.501520] ffff88082d222f10 ffff88082d222f10 0000000000000000 0000000000000000
[18309.510505] Call Trace:
[18309.513916] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.520173] [<ffffffffa0caaee5>] obd_zombie_impexp_thread+0xd5/0x1c0 [obdclass]
[18309.529122] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18309.536039] [<ffffffffa0caae10>] ? obd_zombie_impexp_cull+0x930/0x930 [obdclass]
[18309.545079] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.551190] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.559160] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.565876] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.573841] ptlrpc_hr00_000 S 00000000ffffffff 0 3422 2 0x00000080
[18309.582427] ffff880824bd7dc0 0000000000000046 ffff88082d223ec0 ffff880824bd7fd8
[18309.591435] ffff880824bd7fd8 ffff880824bd7fd8 ffff88082d223ec0 ffff88040fc17608
[18309.600436] 0000000000000000 ffff88082d223ec0 ffff88040fc17600 00000000ffffffff
[18309.609433] Call Trace:
[18309.612853] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.619155] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18309.627035] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18309.634137] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18309.641061] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18309.650195] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.656322] [<ffffffff810bfde3>] ? finish_task_switch+0x53/0x180
[18309.663820] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.671788] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.678504] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.686468] ptlrpc_hr00_001 S 00000000ffffffff 0 3423 2 0x00000080
[18309.695044] ffff8808283abdc0 0000000000000046 ffff88082d224e70 ffff8808283abfd8
[18309.704045] ffff8808283abfd8 ffff8808283abfd8 ffff88082d224e70 ffff88040fc17640
[18309.713046] 0000000000000000 ffff88082d224e70 ffff88040fc17638 00000000ffffffff
[18309.722043] Call Trace:
[18309.725459] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.731725] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18309.739612] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18309.746706] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18309.753639] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18309.762782] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.768909] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.776892] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.783609] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.791589] ptlrpc_hr00_002 S 00000000ffffffff 0 3424 2 0x00000080
[18309.800171] ffff8808262cbdc0 0000000000000046 ffff88082d225e20 ffff8808262cbfd8
[18309.809172] ffff8808262cbfd8 ffff8808262cbfd8 ffff88082d225e20 ffff88040fc17678
[18309.818168] 0000000000000000 ffff88082d225e20 ffff88040fc17670 00000000ffffffff
[18309.827167] Call Trace:
[18309.830590] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.836858] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18309.844746] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18309.851854] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18309.858794] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18309.867946] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.874077] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.882045] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.888752] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.896725] ptlrpc_hr00_003 S 00000000ffffffff 0 3425 2 0x00000080
[18309.905299] ffff88082460fdc0 0000000000000046 ffff88082d226dd0 ffff88082460ffd8
[18309.914307] ffff88082460ffd8 ffff88082460ffd8 ffff88082d226dd0 ffff88040fc176b0
[18309.923309] 0000000000000000 ffff88082d226dd0 ffff88040fc176a8 00000000ffffffff
[18309.932319] Call Trace:
[18309.935733] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18309.941978] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18309.949866] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18309.956971] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18309.963896] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18309.973031] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18309.979155] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18309.987128] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18309.993842] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.001818] ptlrpc_hr00_004 S 00000000ffffffff 0 3426 2 0x00000080
[18310.010418] ffff88082572fdc0 0000000000000046 ffff88082d221f60 ffff88082572ffd8
[18310.019415] ffff88082572ffd8 ffff88082572ffd8 ffff88082d221f60 ffff88040fc176e8
[18310.028417] 0000000000000000 ffff88082d221f60 ffff88040fc176e0 00000000ffffffff
[18310.037415] Call Trace:
[18310.040829] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.047084] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.054964] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.062072] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.069009] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.078137] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.084260] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.092231] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.098953] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.106932] ptlrpc_hr00_005 S 00000000ffffffff 0 3427 2 0x00000080
[18310.115528] ffff880825a67dc0 0000000000000046 ffff88082d220fb0 ffff880825a67fd8
[18310.124542] ffff880825a67fd8 ffff880825a67fd8 ffff88082d220fb0 ffff88040fc17720
[18310.133551] 0000000000000000 ffff88082d220fb0 ffff88040fc17718 00000000ffffffff
[18310.142558] Call Trace:
[18310.145978] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.152228] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.160101] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.167197] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.174121] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.183254] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.189394] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.197386] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.204092] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.212062] ptlrpc_hr00_006 S 00000000ffffffff 0 3428 2 0x00000080
[18310.220653] ffff88081e7abdc0 0000000000000046 ffff8808297f8000 ffff88081e7abfd8
[18310.229661] ffff88081e7abfd8 ffff88081e7abfd8 ffff8808297f8000 ffff88040fc17758
[18310.238662] 0000000000000000 ffff8808297f8000 ffff88040fc17750 00000000ffffffff
[18310.247665] Call Trace:
[18310.251085] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.257336] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.265204] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.272299] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.279217] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.288350] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.294474] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.302443] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.309157] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.317136] ptlrpc_hr00_007 S 00000000ffffffff 0 3429 2 0x00000080
[18310.325727] ffff88082a3b3dc0 0000000000000046 ffff8808297f8fb0 ffff88082a3b3fd8
[18310.334729] ffff88082a3b3fd8 ffff88082a3b3fd8 ffff8808297f8fb0 ffff88040fc17790
[18310.343744] 0000000000000000 ffff8808297f8fb0 ffff88040fc17788 00000000ffffffff
[18310.352756] Call Trace:
[18310.356180] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.362430] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.370302] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.377404] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.384334] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.393473] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.399606] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.407579] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.414278] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.422251] ptlrpc_hr01_000 S 00000000ffffffff 0 3430 2 0x00000080
[18310.430834] ffff880829a87dc0 0000000000000046 ffff8808297f9f60 ffff880829a87fd8
[18310.439828] ffff880829a87fd8 ffff880829a87fd8 ffff8808297f9f60 ffff88082abac208
[18310.448842] 0000000000000000 ffff8808297f9f60 ffff88082abac200 00000000ffffffff
[18310.457847] Call Trace:
[18310.461265] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.467514] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.475387] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.482496] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.489409] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.498541] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.504664] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.512643] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.519356] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.527324] ptlrpc_hr01_001 S 00000000ffffffff 0 3431 2 0x00000080
[18310.535925] ffff880823ed3dc0 0000000000000046 ffff8808297faf10 ffff880823ed3fd8
[18310.544935] ffff880823ed3fd8 ffff880823ed3fd8 ffff8808297faf10 ffff88082abac240
[18310.553934] 0000000000000000 ffff8808297faf10 ffff88082abac238 00000000ffffffff
[18310.562944] Call Trace:
[18310.566362] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.572610] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.580485] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.587581] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.594512] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.603653] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.609773] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.617745] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.624446] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.632416] ptlrpc_hr01_002 S 00000000ffffffff 0 3432 2 0x00000080
[18310.641002] ffff88082af4fdc0 0000000000000046 ffff8808297fbec0 ffff88082af4ffd8
[18310.649996] ffff88082af4ffd8 ffff88082af4ffd8 ffff8808297fbec0 ffff88082abac278
[18310.658986] 0000000000000000 ffff8808297fbec0 ffff88082abac270 00000000ffffffff
[18310.667994] Call Trace:
[18310.671400] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.677658] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.685537] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.692644] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.699568] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.708709] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.714843] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.722828] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.729532] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.737514] ptlrpc_hr01_003 S 00000000ffffffff 0 3433 2 0x00000080
[18310.746098] ffff88082aefbdc0 0000000000000046 ffff8808297fce70 ffff88082aefbfd8
[18310.755101] ffff88082aefbfd8 ffff88082aefbfd8 ffff8808297fce70 ffff88082abac2b0
[18310.764097] 0000000000000000 ffff8808297fce70 ffff88082abac2a8 00000000ffffffff
[18310.773097] Call Trace:
[18310.776502] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.782755] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.790636] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.797745] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.804665] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.813813] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.819948] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.827932] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.834636] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.842606] ptlrpc_hr01_004 S 00000000ffffffff 0 3434 2 0x00000080
[18310.851181] ffff88082a547dc0 0000000000000046 ffff8808297fde20 ffff88082a547fd8
[18310.860164] ffff88082a547fd8 ffff88082a547fd8 ffff8808297fde20 ffff88082abac2e8
[18310.869161] 0000000000000000 ffff8808297fde20 ffff88082abac2e0 00000000ffffffff
[18310.878167] Call Trace:
[18310.881573] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.887824] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18310.895698] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18310.902798] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18310.909727] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18310.918861] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18310.925005] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.932983] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18310.939694] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18310.947672] ptlrpc_hr01_005 S 00000000ffffffff 0 3435 2 0x00000080
[18310.956249] ffff880824727dc0 0000000000000046 ffff8808297fedd0 ffff880824727fd8
[18310.965239] ffff880824727fd8 ffff880824727fd8 ffff8808297fedd0 ffff88082abac320
[18310.974224] 0000000000000000 ffff8808297fedd0 ffff88082abac318 00000000ffffffff
[18310.983218] Call Trace:
[18310.986628] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18310.992869] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18311.000741] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18311.007850] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.014774] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18311.023919] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.030042] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.038012] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.044725] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.052696] ptlrpc_hr01_006 S 00000000ffffffff 0 3436 2 0x00000080
[18311.061272] ffff88083f893dc0 0000000000000046 ffff880824290000 ffff88083f893fd8
[18311.070278] ffff88083f893fd8 ffff88083f893fd8 ffff880824290000 ffff88082abac358
[18311.079256] 0000000000000000 ffff880824290000 ffff88082abac350 00000000ffffffff
[18311.088245] Call Trace:
[18311.091663] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.097918] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18311.105791] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18311.112885] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.119812] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18311.128958] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.135071] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.143040] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.149753] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.157730] ptlrpc_hr01_007 S 00000000ffffffff 0 3437 2 0x00000080
[18311.166306] ffff88081e7a3dc0 0000000000000046 ffff880824290fb0 ffff88081e7a3fd8
[18311.175304] ffff88081e7a3fd8 ffff88081e7a3fd8 ffff880824290fb0 ffff88082abac390
[18311.184300] 0000000000000000 ffff880824290fb0 ffff88082abac388 00000000ffffffff
[18311.193297] Call Trace:
[18311.196720] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.202975] [<ffffffffa0f09a25>] ptlrpc_hr_main+0x675/0x8f0 [ptlrpc]
[18311.210847] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18311.217964] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.224888] [<ffffffffa0f093b0>] ? ptlrpc_svcpt_stop_threads+0x590/0x590 [ptlrpc]
[18311.234023] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.240149] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.248119] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.254824] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.262799] kiblnd_connd S 000000000000f888 0 3447 2 0x00000080
[18311.271392] ffff88082a27bd60 0000000000000046 ffff880824291f60 ffff88082a27bfd8
[18311.280385] ffff88082a27bfd8 ffff88082a27bfd8 ffff880824291f60 ffff8804b9740000
[18311.289384] ffff88082a27bd90 000000010112d378 ffff8804b9740000 000000000000f888
[18311.298382] Call Trace:
[18311.301799] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.308032] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18311.315219] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18311.322620] [<ffffffffa0b5937c>] kiblnd_connd+0x60c/0x900 [ko2iblnd]
[18311.330489] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.337394] [<ffffffffa0b58d70>] ? kiblnd_check_conns+0x840/0x840 [ko2iblnd]
[18311.346041] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.352161] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.360122] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.366827] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.374787] kiblnd_sd_00_00 S ffff88040fb03ee0 0 3448 2 0x00000080
[18311.383363] ffff88082af9fd98 0000000000000046 ffff880824292f10 ffff88082af9ffd8
[18311.392363] ffff88082af9ffd8 ffff88082af9ffd8 ffff880824292f10 ffff88040fb03ec0
[18311.401370] 0000000000000246 ffff88040fb03ec8 ffff880824292f10 ffff88040fb03ee0
[18311.410348] Call Trace:
[18311.413760] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.419973] [<ffffffffa0b59cdc>] kiblnd_scheduler+0x32c/0x1130 [ko2iblnd]
[18311.428324] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18311.435805] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18311.442896] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.449795] [<ffffffffa0b599b0>] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd]
[18311.457960] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.464078] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.472037] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.478738] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.486689] kiblnd_sd_00_01 S ffff88040fb03ee0 0 3449 2 0x00000080
[18311.495251] ffff88081e7e3d98 0000000000000046 ffff880824293ec0 ffff88081e7e3fd8
[18311.504237] ffff88081e7e3fd8 ffff88081e7e3fd8 ffff880824293ec0 ffff88040fb03ec0
[18311.513221] 0000000000000246 ffff88040fb03ec8 ffff880824293ec0 ffff88040fb03ee0
[18311.522195] Call Trace:
[18311.525583] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.531802] [<ffffffffa0b59cdc>] kiblnd_scheduler+0x32c/0x1130 [ko2iblnd]
[18311.540149] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18311.547616] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.554509] [<ffffffffa0b599b0>] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd]
[18311.562663] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.568760] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.576712] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.583407] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.591350] kiblnd_sd_00_02 S ffff88040fb03ee0 0 3450 2 0x00000080
[18311.599924] ffff880826207d98 0000000000000046 ffff880824294e70 ffff880826207fd8
[18311.608907] ffff880826207fd8 ffff880826207fd8 ffff880824294e70 ffff88040fb03ec0
[18311.617899] 0000000000000292 ffff88040fb03ec8 ffff880824294e70 ffff88040fb03ee0
[18311.626887] Call Trace:
[18311.630267] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.636471] [<ffffffffa0b59cdc>] kiblnd_scheduler+0x32c/0x1130 [ko2iblnd]
[18311.644825] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18311.652289] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18311.659354] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.666228] [<ffffffffa0b599b0>] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd]
[18311.674368] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.680466] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.688410] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.695102] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.703047] kiblnd_sd_00_03 S ffff88040fb03ee0 0 3451 2 0x00000080
[18311.711614] ffff88082bff3d98 0000000000000046 ffff880824295e20 ffff88082bff3fd8
[18311.720592] ffff88082bff3fd8 ffff88082bff3fd8 ffff880824295e20 ffff88040fb03ec0
[18311.729585] 0000000000000292 ffff88040fb03ec8 ffff880824295e20 ffff88040fb03ee0
[18311.738561] Call Trace:
[18311.741945] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.748142] [<ffffffffa0b59cdc>] kiblnd_scheduler+0x32c/0x1130 [ko2iblnd]
[18311.756486] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18311.763948] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18311.771019] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.777898] [<ffffffffa0b599b0>] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd]
[18311.786042] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.792139] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.800080] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.806773] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.814725] kiblnd_sd_01_00 S ffff88083f90b860 0 3452 2 0x00000080
[18311.823275] ffff88083f76bd98 0000000000000046 ffff880824296dd0 ffff88083f76bfd8
[18311.832249] ffff88083f76bfd8 ffff88083f76bfd8 ffff880824296dd0 ffff88083f90b840
[18311.841225] 0000000000000246 ffff88083f90b848 ffff880824296dd0 ffff88083f90b860
[18311.850194] Call Trace:
[18311.853568] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.859772] [<ffffffffa0b59cdc>] kiblnd_scheduler+0x32c/0x1130 [ko2iblnd]
[18311.868108] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18311.875571] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18311.882642] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18311.889517] [<ffffffffa0b599b0>] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd]
[18311.897658] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18311.903756] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.911707] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18311.918388] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18311.926323] kiblnd_sd_01_01 S ffff88083f90b860 0 3453 2 0x00000080
[18311.934902] ffff880829c8bd98 0000000000000046 ffff880827730000 ffff880829c8bfd8
[18311.943879] ffff880829c8bfd8 ffff880829c8bfd8 ffff880827730000 ffff88083f90b840
[18311.952857] 0000000000000246 ffff88083f90b848 ffff880827730000 ffff88083f90b860
[18311.961821] Call Trace:
[18311.965201] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18311.971404] [<ffffffffa0b59cdc>] kiblnd_scheduler+0x32c/0x1130 [ko2iblnd]
[18311.979737] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18311.987198] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18311.994258] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18312.001122] [<ffffffffa0b599b0>] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd]
[18312.009257] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.015355] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.023298] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.029991] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.037948] kiblnd_sd_01_02 S ffff88083f90b860 0 3454 2 0x00000080
[18312.046517] ffff8808290cfd98 0000000000000046 ffff880827730fb0 ffff8808290cffd8
[18312.055503] ffff8808290cffd8 ffff8808290cffd8 ffff880827730fb0 ffff88083f90b840
[18312.064474] 0000000000000292 ffff88083f90b848 ffff880827730fb0 ffff88083f90b860
[18312.073419] Call Trace:
[18312.076792] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.082993] [<ffffffffa0b59cdc>] kiblnd_scheduler+0x32c/0x1130 [ko2iblnd]
[18312.091325] [<ffffffff81033669>] ? sched_clock+0x9/0x10
[18312.097917] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18312.104998] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18312.111863] [<ffffffffa0b599b0>] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd]
[18312.119998] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.126089] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.134032] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.140706] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.148651] kiblnd_sd_01_03 S ffff88083f90b860 0 3455 2 0x00000080
[18312.157197] ffff880824303d98 0000000000000046 ffff880827731f60 ffff880824303fd8
[18312.166176] ffff880824303fd8 ffff880824303fd8 ffff880827731f60 ffff88083f90b840
[18312.175146] 0000000000000246 ffff88083f90b848 ffff880827731f60 ffff88083f90b860
[18312.184098] Call Trace:
[18312.187477] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.193676] [<ffffffffa0b59cdc>] kiblnd_scheduler+0x32c/0x1130 [ko2iblnd]
[18312.202018] [<ffffffff810cf22e>] ? dequeue_task_fair+0x41e/0x660
[18312.209476] [<ffffffff810c8345>] ? sched_clock_cpu+0x85/0xc0
[18312.216539] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18312.223411] [<ffffffffa0b599b0>] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd]
[18312.231546] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.237650] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.245599] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.252267] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.260199] router_checker S 000000000000f888 0 3456 2 0x00000080
[18312.268766] ffff880826f03d20 0000000000000046 ffff880827732f10 ffff880826f03fd8
[18312.277739] ffff880826f03fd8 ffff880826f03fd8 ffff880827732f10 ffff8801795fc000
[18312.286724] ffff880826f03d50 000000010112d6b4 ffff8801795fc000 000000000000f888
[18312.295701] Call Trace:
[18312.299074] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.305272] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18312.312430] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18312.319785] [<ffffffff810b1786>] ? prepare_to_wait+0x56/0x90
[18312.326882] [<ffffffffa0c301b7>] lnet_router_checker+0x527/0x890 [lnet]
[18312.335016] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18312.342179] [<ffffffffa0c2fc90>] ? lnet_notify_locked+0x240/0x240 [lnet]
[18312.350401] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.356491] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.364431] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.371103] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.379046] ptlrpcd_rcv S 000000000000f888 0 3457 2 0x00000080
[18312.387601] ffff880826f07cf0 0000000000000046 ffff880827733ec0 ffff880826f07fd8
[18312.396567] ffff880826f07fd8 ffff880826f07fd8 ffff880827733ec0 ffff880179d4c000
[18312.405529] ffff880826f07d20 000000010112d783 ffff880179d4c000 000000000000f888
[18312.414494] Call Trace:
[18312.417880] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.424069] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18312.431228] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18312.438616] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18312.445777] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18312.452680] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18312.460616] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.466717] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.474659] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.481331] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.489264] ptlrpcd_00_00 S 000000000000f888 0 3458 2 0x00000080
[18312.497819] ffff880826f0bcf0 0000000000000046 ffff880827734e70 ffff880826f0bfd8
[18312.506812] ffff880826f0bfd8 ffff880826f0bfd8 ffff880827734e70 ffff8801795fc000
[18312.515768] ffff880826f0bd20 000000010112d7be ffff8801795fc000 000000000000f888
[18312.524749] Call Trace:
[18312.528127] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.534317] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18312.541484] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18312.548867] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18312.556026] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18312.562920] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18312.570857] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.576947] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.584882] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.591545] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.599480] ptlrpcd_00_01 S 000000000000f888 0 3459 2 0x00000080
[18312.608030] ffff880826f0fcf0 0000000000000046 ffff880827735e20 ffff880826f0ffd8
[18312.616995] ffff880826f0ffd8 ffff880826f0ffd8 ffff880827735e20 ffff880179068000
[18312.625957] ffff880826f0fd20 000000010112d7bf ffff880179068000 000000000000f888
[18312.634931] Call Trace:
[18312.638298] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.644490] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18312.651646] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18312.659025] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18312.666178] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18312.673062] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18312.680998] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.687080] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.695008] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.701680] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.709607] ptlrpcd_00_02 S 000000000000f888 0 3460 2 0x00000080
[18312.718144] ffff880825a53cf0 0000000000000046 ffff880827736dd0 ffff880825a53fd8
[18312.727109] ffff880825a53fd8 ffff880825a53fd8 ffff880827736dd0 ffff8801796e4000
[18312.736074] ffff880825a53d20 000000010112d7bd ffff8801796e4000 000000000000f888
[18312.745031] Call Trace:
[18312.748397] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.754590] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18312.761743] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18312.769100] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18312.776248] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18312.783121] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18312.791058] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.797139] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.805061] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.811722] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.819658] ptlrpcd_00_03 S 000000000000f888 0 3461 2 0x00000080
[18312.828197] ffff880825a57cf0 0000000000000046 ffff880825a58000 ffff880825a57fd8
[18312.837148] ffff880825a57fd8 ffff880825a57fd8 ffff880825a58000 ffff8801796b8000
[18312.846107] ffff880825a57d20 000000010112d7bc ffff8801796b8000 000000000000f888
[18312.855054] Call Trace:
[18312.858422] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.864605] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18312.871748] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18312.879120] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18312.886261] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18312.893150] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18312.901062] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18312.907134] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.915064] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18312.921730] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18312.929664] ptlrpcd_00_04 S 000000000000f888 0 3462 2 0x00000080
[18312.938217] ffff880824793cf0 0000000000000046 ffff880825a58fb0 ffff880824793fd8
[18312.947183] ffff880824793fd8 ffff880824793fd8 ffff880825a58fb0 ffff880179674000
[18312.956146] ffff880824793d20 000000010112d809 ffff880179674000 000000000000f888
[18312.965104] Call Trace:
[18312.968472] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18312.974674] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18312.981825] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18312.989188] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18312.996336] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.003218] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.011146] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.017223] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.025150] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.031818] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.039749] ptlrpcd_00_05 S 000000000000f888 0 3463 2 0x00000080
[18313.048287] ffff880824797cf0 0000000000000046 ffff880825a59f60 ffff880824797fd8
[18313.057251] ffff880824797fd8 ffff880824797fd8 ffff880825a59f60 ffff880179644000
[18313.066218] ffff880824797d20 000000010112d848 ffff880179644000 000000000000f888
[18313.075174] Call Trace:
[18313.078557] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.084752] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.091907] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.099274] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.106421] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.113302] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.121232] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.127306] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.135236] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.141892] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.149829] ptlrpcd_00_06 S 000000000000f888 0 3464 2 0x00000080
[18313.158359] ffff88082479bcf0 0000000000000046 ffff880825a5af10 ffff88082479bfd8
[18313.167325] ffff88082479bfd8 ffff88082479bfd8 ffff880825a5af10 ffff8801795fc000
[18313.176271] ffff88082479bd20 000000010112d859 ffff8801795fc000 000000000000f888
[18313.185221] Call Trace:
[18313.188591] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.194777] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.201931] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.209290] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.216437] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.223318] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.231245] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.237323] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.245252] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.251916] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.259845] ptlrpcd_00_07 S 000000000000f888 0 3465 2 0x00000080
[18313.268383] ffff88082479fcf0 0000000000000046 ffff880825a5bec0 ffff88082479ffd8
[18313.277338] ffff88082479ffd8 ffff88082479ffd8 ffff880825a5bec0 ffff88017971c000
[18313.286269] ffff88082479fd20 000000010112dba5 ffff88017971c000 000000000000f888
[18313.295230] Call Trace:
[18313.298596] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.304795] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.311929] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.319292] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.326440] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.333321] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.341243] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.347320] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.355246] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.361923] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.369852] ptlrpcd_00_08 S 000000000000f888 0 3466 2 0x00000080
[18313.378388] ffff88081bab3cf0 0000000000000046 ffff880825a5ce70 ffff88081bab3fd8
[18313.387348] ffff88081bab3fd8 ffff88081bab3fd8 ffff880825a5ce70 ffff880179130000
[18313.396293] ffff88081bab3d20 000000010112dba5 ffff880179130000 000000000000f888
[18313.405220] Call Trace:
[18313.408591] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.414776] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.421932] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.429297] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.436452] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.443335] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.451261] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.457338] [<ffffffff810bfde3>] ? finish_task_switch+0x53/0x180
[18313.464787] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.472720] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.479383] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.487309] ptlrpcd_00_09 S 000000000000f888 0 3467 2 0x00000080
[18313.495871] ffff88081bab7cf0 0000000000000046 ffff880825a5de20 ffff88081bab7fd8
[18313.504842] ffff88081bab7fd8 ffff88081bab7fd8 ffff880825a5de20 ffff880179644000
[18313.513802] ffff88081bab7d20 000000010112dba4 ffff880179644000 000000000000f888
[18313.522754] Call Trace:
[18313.526119] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.532296] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.539442] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.546811] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.553960] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.560837] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.568772] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.574856] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.582776] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.589442] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.597365] ptlrpcd_00_10 S 000000000000f888 0 3468 2 0x00000080
[18313.605909] ffff88081babbcf0 0000000000000046 ffff880825a5edd0 ffff88081babbfd8
[18313.614877] ffff88081babbfd8 ffff88081babbfd8 ffff880825a5edd0 ffff880179674000
[18313.623840] ffff88081babbd20 000000010112dba4 ffff880179674000 000000000000f888
[18313.632805] Call Trace:
[18313.636178] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.642357] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.649502] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.656879] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.664036] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.670918] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.678847] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.684929] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.692856] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.699528] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.707455] ptlrpcd_00_11 S 000000000000f888 0 3469 2 0x00000080
[18313.715996] ffff88081babfcf0 0000000000000046 ffff880825b10000 ffff88081babffd8
[18313.724952] ffff88081babffd8 ffff88081babffd8 ffff880825b10000 ffff880179674000
[18313.733901] ffff88081babfd20 000000010112dba5 ffff880179674000 000000000000f888
[18313.742863] Call Trace:
[18313.746228] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.752404] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.759551] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.766926] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.774081] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.780971] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.788907] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.794997] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.802929] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.809592] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.817522] ptlrpcd_00_12 S 000000000000f888 0 3470 2 0x00000080
[18313.826093] ffff880825b1bcf0 0000000000000046 ffff880825b10fb0 ffff880825b1bfd8
[18313.835064] ffff880825b1bfd8 ffff880825b1bfd8 ffff880825b10fb0 ffff8801796b8000
[18313.844027] ffff880825b1bd20 000000010112dba5 ffff8801796b8000 000000000000f888
[18313.852991] Call Trace:
[18313.856354] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.862547] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.869685] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.877064] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.884218] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18313.891109] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18313.899042] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18313.905123] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.913057] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18313.919737] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18313.927673] ptlrpcd_00_13 S 000000000000f888 0 3471 2 0x00000080
[18313.936214] ffff880825b1fcf0 0000000000000046 ffff880825b11f60 ffff880825b1ffd8
[18313.945173] ffff880825b1ffd8 ffff880825b1ffd8 ffff880825b11f60 ffff880179d4c000
[18313.954133] ffff880825b1fd20 000000010112dba4 ffff880179d4c000 000000000000f888
[18313.963090] Call Trace:
[18313.966452] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18313.972638] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18313.979779] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18313.987136] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18313.994283] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.001164] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.009087] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.015161] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.023090] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.029759] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.037697] ptlrpcd_00_14 S 000000000000f888 0 3472 2 0x00000080
[18314.046242] ffff880825bd3cf0 0000000000000046 ffff880825b12f10 ffff880825bd3fd8
[18314.055185] ffff880825bd3fd8 ffff880825bd3fd8 ffff880825b12f10 ffff8801795fc000
[18314.064139] ffff880825bd3d20 000000010112dba4 ffff8801795fc000 000000000000f888
[18314.073090] Call Trace:
[18314.076459] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.082646] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.089792] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.097163] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.104305] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.111196] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.119125] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.125208] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.133136] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.139802] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.147729] ptlrpcd_00_15 S 000000000000f888 0 3473 2 0x00000080
[18314.156267] ffff880825bd7cf0 0000000000000046 ffff880825b13ec0 ffff880825bd7fd8
[18314.165229] ffff880825bd7fd8 ffff880825bd7fd8 ffff880825b13ec0 ffff880179644000
[18314.174176] ffff880825bd7d20 000000010112dba6 ffff880179644000 000000000000f888
[18314.183139] Call Trace:
[18314.186518] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.192709] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.199864] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.207233] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.214383] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.221266] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.229193] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.235270] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.243200] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.249868] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.257799] ptlrpcd_01_00 S 000000000000f888 0 3474 2 0x00000080
[18314.266338] ffff880825bdbcf0 0000000000000046 ffff880825b14e70 ffff880825bdbfd8
[18314.275296] ffff880825bdbfd8 ffff880825bdbfd8 ffff880825b14e70 ffff8804b97a4000
[18314.284254] ffff880825bdbd20 000000010112dd50 ffff8804b97a4000 000000000000f888
[18314.293211] Call Trace:
[18314.296586] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.302787] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.309944] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.317310] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.324453] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.331331] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.339264] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.345349] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.353279] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.359944] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.367880] ptlrpcd_01_01 S 000000000000f888 0 3475 2 0x00000080
[18314.376422] ffff880825bdfcf0 0000000000000046 ffff880825b15e20 ffff880825bdffd8
[18314.385381] ffff880825bdffd8 ffff880825bdffd8 ffff880825b15e20 ffff8804b96b8000
[18314.394329] ffff880825bdfd20 000000010112dd50 ffff8804b96b8000 000000000000f888
[18314.403278] Call Trace:
[18314.406659] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.412847] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.420001] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.427362] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.434510] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.441393] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.449322] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.455404] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.463330] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.470006] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.477943] ptlrpcd_01_02 S 000000000000f888 0 3476 2 0x00000080
[18314.486505] ffff88082a463cf0 0000000000000046 ffff880825b16dd0 ffff88082a463fd8
[18314.495456] ffff88082a463fd8 ffff88082a463fd8 ffff880825b16dd0 ffff8804b97a4000
[18314.504417] ffff88082a463d20 000000010112dd90 ffff8804b97a4000 000000000000f888
[18314.513360] Call Trace:
[18314.516740] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.522929] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.530089] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.537455] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.544615] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.551499] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.559419] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.565510] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.573432] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.580111] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.588052] ptlrpcd_01_03 S 000000000000f888 0 3477 2 0x00000080
[18314.596610] ffff88082a467cf0 0000000000000046 ffff88082a468000 ffff88082a467fd8
[18314.605586] ffff88082a467fd8 ffff88082a467fd8 ffff88082a468000 ffff8804b90a4000
[18314.614544] ffff88082a467d20 000000010112df60 ffff8804b90a4000 000000000000f888
[18314.623498] Call Trace:
[18314.626878] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.633072] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.640211] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.647580] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.654740] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.661620] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.669546] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.675629] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.683554] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.690222] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.698147] ptlrpcd_01_04 S 000000000000f888 0 3478 2 0x00000080
[18314.706695] ffff8804b93ebcf0 0000000000000046 ffff88082a468fb0 ffff8804b93ebfd8
[18314.715664] ffff8804b93ebfd8 ffff8804b93ebfd8 ffff88082a468fb0 ffff8804b9688000
[18314.724612] ffff8804b93ebd20 000000010112e090 ffff8804b9688000 000000000000f888
[18314.733567] Call Trace:
[18314.736931] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.743114] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.750254] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.757617] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.764766] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.771653] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.779580] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.785659] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.793582] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.800249] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.808169] ptlrpcd_01_05 S 000000000000f888 0 3479 2 0x00000080
[18314.816740] ffff8804b93efcf0 0000000000000046 ffff88082a469f60 ffff8804b93effd8
[18314.825693] ffff8804b93effd8 ffff8804b93effd8 ffff88082a469f60 ffff8804b9770000
[18314.834656] ffff8804b93efd20 000000010112e090 ffff8804b9770000 000000000000f888
[18314.843617] Call Trace:
[18314.846995] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.853176] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.860319] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.867688] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.874840] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.881716] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.889641] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18314.895723] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.903648] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18314.910309] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18314.918217] ptlrpcd_01_06 S 000000000000f888 0 3480 2 0x00000080
[18314.926773] ffff88082672bcf0 0000000000000046 ffff88082a46af10 ffff88082672bfd8
[18314.935747] ffff88082672bfd8 ffff88082672bfd8 ffff88082a46af10 ffff8804b97a4000
[18314.944699] ffff88082672bd20 000000010112e038 ffff8804b97a4000 000000000000f888
[18314.953652] Call Trace:
[18314.957019] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18314.963209] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18314.970350] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18314.977712] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18314.984867] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18314.991748] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18314.999683] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.005769] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.013698] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.020363] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.028289] ptlrpcd_01_07 S 000000000000f888 0 3481 2 0x00000080
[18315.036846] ffff88082672fcf0 0000000000000046 ffff88082a46bec0 ffff88082672ffd8
[18315.045808] ffff88082672ffd8 ffff88082672ffd8 ffff88082a46bec0 ffff8804b9770000
[18315.054774] ffff88082672fd20 000000010112e048 ffff8804b9770000 000000000000f888
[18315.063738] Call Trace:
[18315.067118] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.073297] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.080439] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.087803] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.094952] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.101842] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.109775] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.115864] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.123809] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.130466] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.138390] ptlrpcd_01_08 S 000000000000f888 0 3482 2 0x00000080
[18315.146948] ffff880827723cf0 0000000000000046 ffff88082a46ce70 ffff880827723fd8
[18315.155924] ffff880827723fd8 ffff880827723fd8 ffff88082a46ce70 ffff8804b97d8000
[18315.164886] ffff880827723d20 000000010112e1c8 ffff8804b97d8000 000000000000f888
[18315.173839] Call Trace:
[18315.177205] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.183388] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.190541] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.197917] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.205073] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.211951] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.219886] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.225969] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.233897] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.240572] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.248504] ptlrpcd_01_09 S 000000000000f888 0 3483 2 0x00000080
[18315.257056] ffff880827727cf0 0000000000000046 ffff88082a46de20 ffff880827727fd8
[18315.266018] ffff880827727fd8 ffff880827727fd8 ffff88082a46de20 ffff8804b91b0000
[18315.274986] ffff880827727d20 000000010112e1c8 ffff8804b91b0000 000000000000f888
[18315.283947] Call Trace:
[18315.287327] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.293517] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.300662] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.308035] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.315185] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.322081] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.330010] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.336096] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.344033] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.350696] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.358621] ptlrpcd_01_10 S 000000000000f888 0 3484 2 0x00000080
[18315.367164] ffff88082772bcf0 0000000000000046 ffff88082a46edd0 ffff88082772bfd8
[18315.376127] ffff88082772bfd8 ffff88082772bfd8 ffff88082a46edd0 ffff8804b97a4000
[18315.385083] ffff88082772bd20 000000010112e3c8 ffff8804b97a4000 000000000000f888
[18315.394037] Call Trace:
[18315.397406] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.403597] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.410751] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.418115] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.425269] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.432149] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.440088] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.446163] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.454110] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.460771] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.468692] ptlrpcd_01_11 S 000000000000f888 0 3485 2 0x00000080
[18315.477229] ffff88082772fcf0 0000000000000046 ffff880826e20000 ffff88082772ffd8
[18315.486203] ffff88082772ffd8 ffff88082772ffd8 ffff880826e20000 ffff8804b9068000
[18315.495159] ffff88082772fd20 000000010112e3c8 ffff8804b9068000 000000000000f888
[18315.504112] Call Trace:
[18315.507477] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.513667] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.520815] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.528178] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.535324] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.542201] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.550129] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.556217] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.564147] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.570810] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.578743] ptlrpcd_01_12 S 000000000000f888 0 3486 2 0x00000080
[18315.587294] ffff880826e2bcf0 0000000000000046 ffff880826e20fb0 ffff880826e2bfd8
[18315.596259] ffff880826e2bfd8 ffff880826e2bfd8 ffff880826e20fb0 ffff8804b97d8000
[18315.605205] ffff880826e2bd20 000000010112e3c8 ffff8804b97d8000 000000000000f888
[18315.614153] Call Trace:
[18315.617523] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.623709] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.630850] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.638205] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.645345] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.652218] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.660148] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.666228] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.674152] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.680823] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.688754] ptlrpcd_01_13 S 000000000000f888 0 3487 2 0x00000080
[18315.697296] ffff880826e2fcf0 0000000000000046 ffff880826e21f60 ffff880826e2ffd8
[18315.706253] ffff880826e2ffd8 ffff880826e2ffd8 ffff880826e21f60 ffff8804b96ec000
[18315.715209] ffff880826e2fd20 000000010112e4f8 ffff8804b96ec000 000000000000f888
[18315.724161] Call Trace:
[18315.727531] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.733714] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.740866] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.748229] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.755375] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.762247] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.770186] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.776259] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.784194] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.790859] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.798793] ptlrpcd_01_14 S 000000000000f888 0 3488 2 0x00000080
[18315.807329] ffff880827b63cf0 0000000000000046 ffff880826e22f10 ffff880827b63fd8
[18315.816291] ffff880827b63fd8 ffff880827b63fd8 ffff880826e22f10 ffff8804b90d0000
[18315.825249] ffff880827b63d20 000000010112e510 ffff8804b90d0000 000000000000f888
[18315.834201] Call Trace:
[18315.837570] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.843774] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.850929] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.858290] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.865437] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.872315] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.880240] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.886320] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.894241] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18315.900897] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18315.908825] ptlrpcd_01_15 S 000000000000f888 0 3489 2 0x00000080
[18315.917353] ffff880827b67cf0 0000000000000046 ffff880826e23ec0 ffff880827b67fd8
[18315.926324] ffff880827b67fd8 ffff880827b67fd8 ffff880826e23ec0 ffff8804b9740000
[18315.935278] ffff880827b67d20 000000010112e475 ffff8804b9740000 000000000000f888
[18315.944227] Call Trace:
[18315.947605] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18315.953797] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18315.960947] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18315.968308] [<ffffffffa0f216e8>] ptlrpcd+0x298/0x560 [ptlrpc]
[18315.975456] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18315.982339] [<ffffffffa0f21450>] ? ptlrpcd_check+0x5d0/0x5d0 [ptlrpc]
[18315.990271] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18315.996354] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.004282] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.010959] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.018897] ll_ping S 000000000000f888 0 3490 2 0x00000080
[18316.027450] ffff880827b6bd20 0000000000000046 ffff880826e24e70 ffff880827b6bfd8
[18316.036393] ffff880827b6bfd8 ffff880827b6bfd8 ffff880826e24e70 ffff8801796e4000
[18316.045351] ffff880827b6bd50 0000000101132dac ffff8801796e4000 000000000000f888
[18316.054311] Call Trace:
[18316.057679] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.063875] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18316.071024] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18316.078389] [<ffffffffa0f1457c>] ptlrpc_pinger_main+0x92c/0xbb0 [ptlrpc]
[18316.086618] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18316.093503] [<ffffffffa0f13c50>] ? ptlrpc_obd_ping+0x130/0x130 [ptlrpc]
[18316.101629] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.107728] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.115674] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.122340] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.130273] sptlrpc_gc S 000000000000f888 0 3491 2 0x00000080
[18316.138848] ffff880827b6fd70 0000000000000046 ffff880826e25e20 ffff880827b6ffd8
[18316.147835] ffff880827b6ffd8 ffff880827b6ffd8 ffff880826e25e20 ffff8804b9100000
[18316.156820] ffff880827b6fda0 00000001012ae3bc ffff8804b9100000 000000000000f888
[18316.165786] Call Trace:
[18316.169161] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.175345] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18316.182490] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18316.189850] [<ffffffffa0b90ec1>] ? cfs_block_sigsinv+0x71/0xa0 [libcfs]
[18316.198014] [<ffffffffa0f349a5>] sec_gc_main+0x2e5/0x420 [ptlrpc]
[18316.205573] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18316.212468] [<ffffffffa0f346c0>] ? sptlrpc_gc_add_ctx+0x130/0x130 [ptlrpc]
[18316.220902] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.227008] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.234958] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.241636] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.249586] kmmpd-dm-5 S 000000000000f888 0 3492 2 0x00000080
[18316.258169] ffff880826b67d60 0000000000000046 ffff880826e26dd0 ffff880826b67fd8
[18316.267135] ffff880826b67fd8 ffff880826b67fd8 ffff880826e26dd0 ffff880179130000
[18316.276117] ffff880826b67d90 000000010112e6d4 ffff880179130000 000000000000f888
[18316.285103] Call Trace:
[18316.288479] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.294669] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18316.301837] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18316.309209] [<ffffffffa114ff42>] ? write_mmp_block+0x132/0x170 [ldiskfs]
[18316.317440] [<ffffffff81689ece>] schedule_timeout_interruptible+0x1e/0x20
[18316.325789] [<ffffffffa1150206>] kmmpd+0x216/0x430 [ldiskfs]
[18316.332862] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18316.339649] [<ffffffffa114fff0>] ? __dump_mmp_msg+0x70/0x70 [ldiskfs]
[18316.347611] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.353710] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.361653] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.368338] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.376275] jbd2/dm-5-8 S ffff88042b897800 0 3493 2 0x00000080
[18316.384856] ffff88082bcc7e30 0000000000000046 ffff88082d210000 ffff88082bcc7fd8
[18316.393843] ffff88082bcc7fd8 ffff88082bcc7fd8 ffff88082d210000 ffff88042b897828
[18316.402822] ffff88042b897be8 ffff88042b8978a8 ffff88082bcc7e80 ffff88042b897800
[18316.411796] Call Trace:
[18316.415183] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.421383] [<ffffffffa0268fd3>] kjournald2+0x203/0x260 [jbd2]
[18316.428649] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18316.435831] [<ffffffffa0268dd0>] ? commit_timeout+0x10/0x10 [jbd2]
[18316.443480] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.449575] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.457503] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.464192] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.472138] ldiskfs-rsv-con S ffff88082ab2e800 0 3494 2 0x00000080
[18316.480706] ffff88082c943e08 0000000000000046 ffff88082d210fb0 ffff88082c943fd8
[18316.489675] ffff88082c943fd8 ffff88082c943fd8 ffff88082d210fb0 ffff88082d210fb0
[18316.498637] ffff88082ab2e800 ffff88082aba9e80 ffff88082ab2e830 ffff88082ab2e800
[18316.507612] Call Trace:
[18316.511001] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.517189] [<ffffffff810a9069>] rescuer_thread+0x359/0x460
[18316.524172] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18316.530943] [<ffffffff810a8d10>] ? manage_workers.isra.24+0x2d0/0x2d0
[18316.538888] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.544976] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.552915] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.559589] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.567520] ldlm_cb00_000 S ffff88042d674c00 0 3498 2 0x00000080
[18316.576068] ffff8808253dfd38 0000000000000046 ffff88082d211f60 ffff8808253dffd8
[18316.585053] ffff8808253dffd8 ffff8808253dffd8 ffff88082d211f60 0000000000000000
[18316.594033] ffff88082d211f60 ffff8804036c0080 ffff88042d674c68 ffff88042d674c00
[18316.603011] Call Trace:
[18316.606390] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.612612] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18316.620753] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18316.627654] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18316.635302] [<ffffffff81029569>] ? __switch_to+0xd9/0x4c0
[18316.642076] [<ffffffff8168bdc0>] ? __schedule+0x3b0/0x990
[18316.648882] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18316.657807] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.663910] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.671858] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.678527] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.686469] lc_watchdogd S ffff88082d212f10 0 3499 2 0x00000080
[18316.695048] ffff8808253e3e18 0000000000000046 ffff88082d212f10 ffff8808253e3fd8
[18316.704037] ffff8808253e3fd8 ffff8808253e3fd8 ffff88082d212f10 ffffffffa0bbaed0
[18316.713023] 0000000000000000 ffffffffa0b9e910 0000000000000000 ffff88082d212f10
[18316.722025] Call Trace:
[18316.725422] [<ffffffffa0b9e910>] ? lc_watchdog_add+0x790/0x790 [libcfs]
[18316.733585] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.739802] [<ffffffffa0b9f0a9>] lcw_dispatch_main+0x799/0x990 [libcfs]
[18316.747959] [<ffffffff810b1b20>] ? wake_up_atomic_t+0x30/0x30
[18316.755166] [<ffffffffa0b9e910>] ? lc_watchdog_add+0x790/0x790 [libcfs]
[18316.763320] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.769433] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.777387] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.784097] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.792069] ldlm_cb00_001 S ffff88042d674c00 0 3500 2 0x00000080
[18316.800654] ffff88082bf6fd38 0000000000000046 ffff88082d213ec0 ffff88082bf6ffd8
[18316.809645] ffff88082bf6ffd8 ffff88082bf6ffd8 ffff88082d213ec0 0000000000000000
[18316.818646] ffff88082d213ec0 ffff8804036c0580 ffff88042d674c68 ffff88042d674c00
[18316.827633] Call Trace:
[18316.831037] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.837280] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18316.845446] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18316.852380] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18316.860059] [<ffffffff81029569>] ? __switch_to+0xd9/0x4c0
[18316.866898] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18316.875844] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.881986] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.889975] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18316.896694] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.904688] ldlm_cb01_000 S ffff880827238000 0 3501 2 0x00000080
[18316.913280] ffff88082bed3d38 0000000000000046 ffff88082d214e70 ffff88082bed3fd8
[18316.922304] ffff88082bed3fd8 ffff88082bed3fd8 ffff88082d214e70 0000000000000000
[18316.931322] ffff88082d214e70 ffff88082ca28b80 ffff880827238068 ffff880827238000
[18316.940324] Call Trace:
[18316.943748] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18316.950022] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18316.958214] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18316.965159] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18316.972875] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18316.981845] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18316.987994] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18316.995980] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.002708] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.010695] ldlm_cb01_001 S ffff880827238000 0 3502 2 0x00000080
[18317.019289] ffff88082aaf3d38 0000000000000046 ffff88082d215e20 ffff88082aaf3fd8
[18317.028317] ffff88082aaf3fd8 ffff88082aaf3fd8 ffff88082d215e20 0000000000000000
[18317.037345] ffff88082d215e20 ffff88082ca28980 ffff880827238068 ffff880827238000
[18317.046372] Call Trace:
[18317.049805] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.056082] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18317.064268] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.071228] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18317.078945] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18317.087908] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.094055] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.102042] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.108756] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.116744] ldlm_cn00_000 S ffff88042d677c00 0 3503 2 0x00000080
[18317.125349] ffff88082c7dfd38 0000000000000046 ffff88082d216dd0 ffff88082c7dffd8
[18317.134371] ffff88082c7dffd8 ffff88082c7dffd8 ffff88082d216dd0 0000000000000000
[18317.143397] ffff88082d216dd0 ffff8804036c0600 ffff88042d677c68 ffff88042d677c00
[18317.152423] Call Trace:
[18317.155857] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.162115] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18317.170295] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.177237] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18317.184950] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18317.193901] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.200050] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.208042] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.214764] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.222758] ldlm_cn00_001 S ffff88042d677c00 0 3504 2 0x00000080
[18317.231348] ffff88083f6ebd38 0000000000000046 ffff880826af8000 ffff88083f6ebfd8
[18317.240370] ffff88083f6ebfd8 ffff88083f6ebfd8 ffff880826af8000 0000000000000000
[18317.249387] ffff880826af8000 ffff8804036c0000 ffff88042d677c68 ffff88042d677c00
[18317.258397] Call Trace:
[18317.261837] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.268114] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18317.276291] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.283246] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18317.290968] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18317.299933] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.306087] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.314078] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.320806] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.328794] ldlm_cn00_002 S ffff88042d677c00 0 3505 2 0x00000080
[18317.337379] ffff880826a83d38 0000000000000046 ffff880826af8fb0 ffff880826a83fd8
[18317.346401] ffff880826a83fd8 ffff880826a83fd8 ffff880826af8fb0 0000000000000000
[18317.355418] ffff880826af8fb0 ffff8804036c0880 ffff88042d677c68 ffff88042d677c00
[18317.364433] Call Trace:
[18317.367875] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.374151] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18317.382339] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.389287] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18317.397009] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18317.405974] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.412130] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.420112] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.426828] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.434823] ldlm_cn01_000 S ffff88082723fc00 0 3506 2 0x00000080
[18317.443421] ffff880826a87d38 0000000000000046 ffff880826af9f60 ffff880826a87fd8
[18317.452440] ffff880826a87fd8 ffff880826a87fd8 ffff880826af9f60 0000000000000000
[18317.461461] ffff880826af9f60 ffff88082ca28800 ffff88082723fc68 ffff88082723fc00
[18317.470471] Call Trace:
[18317.473893] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.480174] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18317.488353] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.495292] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18317.503014] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18317.511979] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.518137] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.526145] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.532856] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.540839] ldlm_cn01_001 S ffff88082723fc00 0 3507 2 0x00000080
[18317.549434] ffff880826a8bd38 0000000000000046 ffff880826afaf10 ffff880826a8bfd8
[18317.558450] ffff880826a8bfd8 ffff880826a8bfd8 ffff880826afaf10 0000000000000000
[18317.567464] ffff880826afaf10 ffff88082ca29200 ffff88082723fc68 ffff88082723fc00
[18317.576469] Call Trace:
[18317.579890] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.586153] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18317.594321] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.601277] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18317.609003] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18317.617953] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.624100] [<ffffffff810bfde3>] ? finish_task_switch+0x53/0x180
[18317.631600] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.639579] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.646298] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.654279] ldlm_cn01_002 S ffff88082723fc00 0 3508 2 0x00000080
[18317.662883] ffff880826a8fd38 0000000000000046 ffff880826afbec0 ffff880826a8ffd8
[18317.671909] ffff880826a8ffd8 ffff880826a8ffd8 ffff880826afbec0 0000000000000000
[18317.680931] ffff880826afbec0 ffff88082ca29480 ffff88082723fc68 ffff88082723fc00
[18317.689952] Call Trace:
[18317.693369] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.699630] [<ffffffffa0f0ae45>] ptlrpc_wait_event+0x325/0x340 [ptlrpc]
[18317.707809] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.714758] [<ffffffffa0f10d9b>] ptlrpc_main+0x80b/0x1dd0 [ptlrpc]
[18317.722467] [<ffffffffa0f10590>] ? ptlrpc_register_service+0xe60/0xe60 [ptlrpc]
[18317.731420] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.737554] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.745533] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.752249] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.760228] ldlm_bl_01 S ffff88082ca291a8 0 3509 2 0x00000080
[18317.768823] ffff880826a93e08 0000000000000046 ffff880826afce70 ffff880826a93fd8
[18317.777824] ffff880826a93fd8 ffff880826a93fd8 ffff880826afce70 ffff88082ca29180
[18317.786846] ffff880826afce70 ffff88082ca291e4 0000000000000000 ffff88082ca291a8
[18317.795872] Call Trace:
[18317.799284] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.805531] [<ffffffffa0eddabd>] ldlm_bl_thread_main+0x42d/0x700 [ptlrpc]
[18317.813901] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.820827] [<ffffffffa0edd690>] ? ldlm_handle_bl_callback+0x410/0x410 [ptlrpc]
[18317.829786] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.835915] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.843894] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.850609] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.858587] ldlm_bl_02 S ffff88082ca291a8 0 3510 2 0x00000080
[18317.867197] ffff880826a97e08 0000000000000046 ffff880826afde20 ffff880826a97fd8
[18317.876225] ffff880826a97fd8 ffff880826a97fd8 ffff880826afde20 ffff88082ca29180
[18317.885241] ffff880826afde20 ffff88082ca291e4 0000000000000000 ffff88082ca291a8
[18317.894242] Call Trace:
[18317.897663] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18317.903923] [<ffffffffa0eddabd>] ldlm_bl_thread_main+0x42d/0x700 [ptlrpc]
[18317.912284] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18317.919216] [<ffffffffa0edd690>] ? ldlm_handle_bl_callback+0x410/0x410 [ptlrpc]
[18317.928194] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18317.934318] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.942294] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18317.949013] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18317.956998] ldlm_elt S 0000000000000000 0 3511 2 0x00000080
[18317.965598] ffff880826a9be00 0000000000000046 ffff880826afedd0 ffff880826a9bfd8
[18317.974616] ffff880826a9bfd8 ffff880826a9bfd8 ffff880826afedd0 ffff880826afedd0
[18317.983619] 0000000000000000 ffffffffa0ed4a00 ffff880826afedd0 0000000000000000
[18317.992625] Call Trace:
[18317.996071] [<ffffffffa0ed4a00>] ? ldlm_bl_to_thread+0x490/0x510 [ptlrpc]
[18318.004451] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18318.010715] [<ffffffffa0ed4c42>] expired_lock_main+0x1c2/0x910 [ptlrpc]
[18318.018900] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18318.025825] [<ffffffffa0ed4a80>] ? ldlm_bl_to_thread+0x510/0x510 [ptlrpc]
[18318.034207] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18318.040342] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18318.048323] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18318.055040] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18318.063031] ldlm_poold S 000000000000f888 0 3512 2 0x00000080
[18318.071641] ffff880826a9fd70 0000000000000046 ffff880827f20000 ffff880826a9ffd8
[18318.080662] ffff880826a9ffd8 ffff880826a9ffd8 ffff880827f20000 ffff8804b9688000
[18318.089672] ffff880826a9fda0 000000010112eec0 ffff8804b9688000 000000000000f888
[18318.098673] Call Trace:
[18318.102096] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18318.108334] [<ffffffff81689d64>] schedule_timeout+0x174/0x2c0
[18318.115538] [<ffffffff81096d60>] ? internal_add_timer+0x70/0x70
[18318.122943] [<ffffffffa0b90ec1>] ? cfs_block_sigsinv+0x71/0xa0 [libcfs]
[18318.131130] [<ffffffffa0ee799b>] ldlm_pools_thread_main+0x23b/0x330 [ptlrpc]
[18318.139797] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18318.146730] [<ffffffffa0ee7760>] ? ldlm_pools_recalc+0x3d0/0x3d0 [ptlrpc]
[18318.155106] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18318.161247] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18318.169236] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18318.175948] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18318.183929] mgs_params_noti S ffff880827f20fb0 0 3513 2 0x00000080
[18318.192498] ffff880827f3bdd8 0000000000000046 ffff880827f20fb0 ffff880827f3bfd8
[18318.201495] ffff880827f3bfd8 ffff880827f3bfd8 ffff880827f20fb0 ffff880827f20fb0
[18318.210495] 0000000200000000 ffff88082abae328 ffff88082abae200 ffff880827f20fb0
[18318.219490] Call Trace:
[18318.222903] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18318.229135] [<ffffffffa1285a40>] mgs_ir_notify+0x1c0/0x2b0 [mgs]
[18318.236619] [<ffffffff810c54c0>] ? wake_up_state+0x20/0x20
[18318.243524] [<ffffffffa1285880>] ? lprocfs_ir_set_state+0x170/0x170 [mgs]
[18318.251877] [<ffffffff810b0a4f>] kthread+0xcf/0xe0
[18318.257990] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18318.265946] [<ffffffff81697318>] ret_from_fork+0x58/0x90
[18318.272654] [<ffffffff810b0980>] ? kthread_create_on_node+0x140/0x140
[18318.280613] ll_mgs_0000 S ffff8803fce6fc00 0 3514 2 0x00000080
[18318.289200] ffff880825af3d38 0000000000000046 ffff880827f21f60 ffff880825af3fd8
[18318.298209] ffff880825af3fd8 ffff880825af3fd8 ffff880827f21f60 0000000000000000
[18318.307221] ffff880827f21f60 ffff8804036c0680 ffff8803fce6fc68 ffff8803fce6fc00
[18318.316223] Call Trace:
[18318.319642] [<ffffffff8168c3c9>] schedule+0x29/0x70
[18318.319669] [<ffffffffa0f0ae45>] [ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct